Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Guidelines to determine the right interface when
integrating with SAP systems
Alaa Karam
Master of Science in Engineering
...
2
Table of Contents
1. Summary...............................................................................................
3
5. Check and Act ..........................................................................................................
4
Guidelines to determine the right interface when
integrating with SAP systems
1. Summary
The purpose of this document is...
5
2. Introduction
2.1 Background and problem
• Applications in today’s system landscape are heavily connected to each
othe...
6
alternative interfaces. At the beginning, when we as integration developers
and integration architects, started to creat...
7
• The sender and/or the receiver system can communicate by other types of
interfaces than just by the File interface.
• ...
8
1. I added the stakeholder and stakeholders’ needs as new component
in the center of the PDCA method. Stakeholders are t...
9
• Clarify and meet their expectations
• At the end you may not satisfy all your stakeholders. The key stakeholders
with ...
10
3.1.1 The requirements of a project
The requirements on SAP integration or on a specific information flow that involves...
11
protocols. Often these systems are designed with security in
mind.
Interoperability According to [23], interoperability...
12
flexibility, a Dynamic Router can be very useful. This router allows
the routing logic to be modified by sending contro...
13
Interoperability is one of the main required capabilities in every enterprise. Projects
with IT components (e.g. purcha...
14
How much data will be transferred per time (day, week and
month) in kilobytes and in business related
numbers/transacti...
15
RFC
sRFC
tRFC
qRFC
BAPI
Figure 2: The RFC types
Synchronous RFC
The first version of RFC is synchronous RFC (sRFC). Thi...
16
o Disadvantages of tRFC
o tRFC processes all LUWs independently of one another. Due to the
amount of activated tRFC pro...
17
o Will only work if there is an active online connection.
o Some programming required to call a BAPI.
We need to explai...
18
o No additional programming required. You just need to set up the
configuration.
• Disadvantages
o Receipt/processing o...
19
• Disadvantages
o The disadvantage of asynchronous communications via ALE is that
only a single return parameter from t...
20
Figure 4: Interface/Message Bridge (Source [4])
Use a Messaging Bridge, a connection between messaging systems, to repl...
21
Figure 5: Guaranteed Delivery (Source [4])
3.2.2.3 Transactional messages
A messaging system, by necessity, uses transa...
22
To explain more about transactional behaviour, we take another example from the
SOA patterns.
According to [22], as per...
23
Figure 7: Event Driven Messaging (Source [4])
3.2.2.5 Synchronous and Asynchronous Communication
Communication between ...
24
3.2.2.7 Asynchronous Communication
According to [25]:
For asynchronous communication, the receiving system does not nec...
25
Figure 9: Asynchronous communication n(Source [25])
3.2.2.8 WebServices (with Reliable Messaging)
According to [26]:
Wh...
26
3.2.2.9 WebServices (without Reliable Messaging)
You can either use the inside-out or outside-in approach to develop a ...
27
The tools to manipulate files (for viewing, deleting, appending, etc) are known by
everyone and built into the operatin...
28
4.1.1 Connectivity to applications
Another important parameter that should be taken into consideration is the
connectiv...
29
The diagram in the following figure, explains how the interfacing mechanisms in SAP
is working. The diagram lists the m...
30
SAP NetWeaver
Application Server
Web
Service
SAP R/3, mySAP ERP
IDoc Inbound/
Outbound Processing
BAPI/
RFC
ALE Inbound...
31
Requirements
Security
Performance
Interface
Maintaina-
bility
Reusability
Common Quality Attributes
Specific Integratio...
32
Table 3: Decision Matrix
Explanations:
Async. Com.: Asynchronous communication;
No = not allowed;
Yes = shall be used;
...
33
4.1.3 Determining which interface method to use when interfacing to SAP
Depending on the requirements on the SAP interf...
34
2. Scenario 2: an application has the possibility to send IDocs to SAP.
3. Scenario 3: an application has the possibili...
35
If assume that the enterprise uses the most popular application integration tool
(BizTalk Server 20XX), the BizTalk Arc...
36
1
2
3
4
5
6
7
8
Figure 14: The path of a message in BizTalk Server
The work breakdown structure (WBS) for an integratio...
37
Provide documentation and contact
Find the contact persons (system owner, system manager) for
the application.
Connecti...
38
4.1.5 Create a budget
The budget will be based on the estimation of all required information flow from the
project. As ...
39
4.2 Do
4.2.1 A Case Study (how to use this document)
In general, it is a big challenge to determine a right interface. ...
40
i. WebService Client/ WebService: Can connect to other systems
with a WebService Client or provides WebService to excha...
41
b. Alternative 2: The second alternative is based on the possibility that
the SAP system has with providing WebServices...
42
d. As we mentioned in the beginning, we assume that the enterprise has
invested previously in an Integration platform (...
43
The disadvantage with involving an EAIP is the performance (the delay
when a message will pass another system “the EAIP...
44
As we mentioned in table 4: WBS for an Information Flow, modifications need to be
identified analysed and estimated.
6....
45
Upcoming SlideShare
Loading in …5
×

Guidelines to determine the right interface when integrating with sap systems version 1.4

20,084 views

Published on

The purpose of this document is to help and guide projects with application integration components to determine the right interface when communicating with SAP systems.

Published in: Data & Analytics

Guidelines to determine the right interface when integrating with sap systems version 1.4

  1. 1. Guidelines to determine the right interface when integrating with SAP systems Alaa Karam Master of Science in Engineering IT Demand Architect/IT Demand Manager at Vattenfall 2014-12-15
  2. 2. 2 Table of Contents 1. Summary.................................................................................................................................. 4 2. Introduction............................................................................................................................. 5 2.1 Background and problem....................................................................................... 5 2.2 Constraints.............................................................................................................. 6 2.3 Purpose................................................................................................................... 7 2.4 Scope ...................................................................................................................... 7 2.5 Audience................................................................................................................. 7 3. The Method ............................................................................................................................. 7 3.1 Stakeholders and Stakeholders’ needs .................................................................. 8 3.1.1 The requirements of a project.............................................................................. 10 3.2 Definitions, Acronyms and Abbreviations............................................................ 14 3.2.1 SAP related Definitions ......................................................................................... 14 3.2.1.1 RFC........................................................................................................... 14 3.2.1.2 BAPI ......................................................................................................... 16 3.2.1.3 IDoc ......................................................................................................... 17 3.2.1.4 ALE........................................................................................................... 18 3.2.2 Other Definitions .................................................................................................. 19 3.2.2.1 Adapters and Interfaces.......................................................................... 19 3.2.2.2 Guaranteed Delivery............................................................................... 20 3.2.2.3 Transactional messages .......................................................................... 21 3.2.2.4 Event driven messaging .......................................................................... 22 3.2.2.5 Synchronous and Asynchronous Communication .................................. 23 3.2.2.6 Synchronous communication ................................................................. 23 3.2.2.7 Asynchronous Communication............................................................... 24 3.2.2.8 WebServices (with Reliable Messaging) ................................................. 25 3.2.2.9 WebServices (without Reliable Messaging)............................................ 26 3.2.2.10SFTP......................................................................................................... 26 3.2.2.11Files ......................................................................................................... 26 4. Plan ........................................................................................................................................ 27 4.1.1 Connectivity to applications ................................................................................. 28 4.1.1.1 How SAP communicates/interfaces with other systems?...................... 28 4.1.2 The decision matrix............................................................................................... 30 4.1.3 Determining which interface method to use when interfacing to SAP................ 33 4.1.3.1 Integration Scenario................................................................................ 33 4.1.4 Estimate the solution............................................................................................ 34 4.1.5 Create a budget..................................................................................................... 38 4.1.6 Plan the implementation...................................................................................... 38 4.2 Do.......................................................................................................................... 39 4.2.1 A Case Study (how to use this document)............................................................ 39
  3. 3. 3 5. Check and Act ........................................................................................................................ 43 6. References ............................................................................................................................. 44
  4. 4. 4 Guidelines to determine the right interface when integrating with SAP systems 1. Summary The purpose of this document is to help and guide projects with application integration components to determine the right interface when communicating with SAP systems. The main input to determine the right interface is the business requirements. These requirements need to be translated into integration requirements and agreed with the business and the IT parts. Depending upon the requirements on the integration flow (e.g. Guaranteed Delivery, Transactional1 messages, the reliability and performance of the interface), the right SAP interface should be selected when the project designs the solution. The project needs to map the requirements in the left side with the accurate interface in the right side, see the following figure: Requirements Security Performance Interface Maintaina- bility Reusability Common Quality Attributes Specific Integration Requirements Guaranteed Delivery Transactionality Ordered delivery WebService ALE BAPI/RFC IDoc File/(S)FTP …. Synchronous communication Message Routing, Traceability, Orchestration,... Asynchronous communication There are several types of SAP interfaces to choose from (see the table). We must consider at least two cases: Case 1: If requirements include Guaranteed Delivery, Transactional messages); the solution have to use either: IDocs, ALE, or WebServices with Reliable Messaging or some types of BAPI/RFC. Case 2: If the requirements are less strict and the consequences of losing messages are not that serious; it is permissible to use File/FTP integration or WebServices without Reliable Messaging. 1 Transactionality: In this scope we define Transactionality as ‘the ability to ensure that a message is conveyed in its entirety’. The whole message or nothing at all is sent.
  5. 5. 5 2. Introduction 2.1 Background and problem • Applications in today’s system landscape are heavily connected to each other. Processes crossing over different companies (e.g. switching electricity providers, Meter Reading and Billing, Supply Chain Management Processes etc.). Information need to be exchanged between the corresponding systems that run and support these processes. • A System (e.g. SAP ERP) is the master source of many information objects (e.g. Products, Product Prices, Invoices, Payments etc.). Other processes and internal/external systems are depended on the information (data) that is stored and owned by the SAP ERP system. In order to feed these systems (e.g. Business Intelligence systems, CRM, SRM, other external and internal systems) with the required data, integration flows between these applications or systems should be developed. • A system like SAP ERP requires to get the right information from other systems (e.g. Meter Reading, CRM, SRM,…) in order to run the processes (e.g. Billing the customers) correctly. • As we see, interoperability is a very important capability and it is required by every enterprise. • Projects have to deliver their outcomes in time and according to the budget and the agreed quality. Unfortunately many projects detect the integration needs when they are in a late phase of the project. Often these projects strive to solve the integration needs without analyzing the business requirements for these integration flows. The main concern of a project manager is to avoid delays in the projects. The consequences of a quick design of these integration flows could be a rework of these flows or more serious consequences (e.g. lose profits, lose reputation) by losing the messages or sending message more than one time. • Often projects do not consider other parameters that characterize a future- proof solution: o Reusability o Transactionality o Maintainability, simplicity of management (e.g. debugging, monitoring, exception handling etc) o Configurability (using Scheduling mechanism) o Adaptability to future needs o Security; transport with secure protocols o Savings in development and maintenance o Compliance with the SAP standard solution (in some cases applications has a SAP certified interface). • It is not an easy issue to determine the right interface for integration between two or more systems, especially when the systems have many
  6. 6. 6 alternative interfaces. At the beginning, when we as integration developers and integration architects, started to create integration flows between our SAP systems with the rest of the world in 2004, we made many mistakes and we made many reworks and redesigns and of course we learned a lot of lessons from these mistakes. I hope that this document will help you to avoid some reworks and help you to take the right design decision when it comes to integration with SAP systems. • Many companies use SAP products as (e.g. their ERP system). In our organization we have a long experience of using SAP systems. I wrote the first version of this document in 2007 as a one page-document and I made several versions of this document in 2008, 2009 and 2010 when I was a solution architect and the responsible for the EAI (Enterprise Application Integration) area in our organization. During these years the document was used by many integration/solution architects and developers to determine the right interface in their projects. • In my opinion, the main tasks of a integration architect is to: o Design an integration solution that is according to the business requirements o Connecting the right systems that contains the right/required information in a right way 2.2 Constraints There are many conditions and constraints that matter when determining the right interface: • The availability of an EAI tool in the organization (e.g. BizTalk Server 20XX, SAP PI 7.X and the existence of SAP adapters to handle IDocs, BAPI and RFC messages). • If the sender and the receiver system belong to different security zones/different networks • If the sender or the receiver system can communicate only via File interface • The SAP version(s) that are installed in the organization. This document is based on general interfaces (e.g.: BAPI/RFC, IDocs, ALE) that are provided by many SAP versions (e.g. SAP R/3, My SAP ERP Edition 2004, SAP ERP 6.0,…). In order to create a right content in this document, I made some assumptions based on the previous points. These assumptions may not be necessarily incorporated in the sender or the receiver systems’ IT-landscape. • The IT-landscape includes an EAI platform. This assumption does not mean that every interface goes via an EAI platform. The document assumes that there are clear definitions, when an EAI platform will be used. The criteria for selection the right interface are not depending on using or not using an integration platform. • The sender and the receiver system belong to one security zone.
  7. 7. 7 • The sender and/or the receiver system can communicate by other types of interfaces than just by the File interface. • The sender and/or the receiver system can implement some or all the general SAP interfaces. 2.3 Purpose • The purpose of this document is to help and guide projects with application integration components to formulate design decisions and determine the right interface when communicating with SAP systems. The main input to determine the right interface is the business requirements. These requirements need to be translated to integration requirements. 2.4 Scope The current document answers the question: • How to determine a suitable interface to use when integrating to SAP? The following topics are not included in the guideline document: o When to use an Enterprise Application Integration (EAI) tool and when it is allowed to not use it. This issue is a matter for a new work (guidelines) in the future. o Different versions of SAP and their impacts on the SAP provided interfaces. o The roadmap of SAP interfaces/integration platform. o The impacts of using custom IDocs, custom BAPIs and custom RFCs when the project does not find a standard IDocs, BAPI or RFC. o Setup and configuration of RFC, IDocs and other connections. o SAP Enterprise Service Oriented Architecture (or Enterprise SOA) 2.5 Audience This document is intended for project managers, developers, business architects, solution architects, software experts, software architects, and anyone responsible or involved in taking design decisions when there are needs for integrating SAP systems with other SAP or non-SAP systems. 3. The Method The method used in this document is based on PDCA (Plan, Do, Check and Act) method. We use this method to gather and agree about requirements, plan and design the solution, implement, test and act if there are any gaps between the requirements and the outcomes of the integration solution. The PDCA cycle is widely used in industry, lines of business to improve processes and as a tool for problem solving.
  8. 8. 8 1. I added the stakeholder and stakeholders’ needs as new component in the center of the PDCA method. Stakeholders are the start point and the end point of every assignment. The main task of a project manager is to: • Ensure the satisfaction of the stakeholders. • Translation of the stakeholders’ needs to understandable requirements in order to ensure a correct delivery 2. Plans will start with scoping the goal, objectives and requirements on the integration solution. 3. Do according to the plan. 4. Check if there are some gaps between the required outcomes and the actual outcomes. 5. We may need to take more actions to correct the result. Plan DoAct Check Stake- holders Needs 1 2 3 4 5 Figure 1: PDCA with stakeholders management 3.1 Stakeholders and Stakeholders’ needs If we go back to our method, we put our stakeholders and their needs in the middle of the PDCA method and we decided to start with the stakeholders. The first step is to identify these stakeholders. Some main principles for managing your stakeholders: • If you have many stakeholders and if there are risks for conflicts between different interests and expectations, then you may need to analyze your stakeholders’ power and influence in your assignment.
  9. 9. 9 • Clarify and meet their expectations • At the end you may not satisfy all your stakeholders. The key stakeholders with the most power and influence are very important to concentrate on. Identifying stakeholders is not an easy task and you as a project manager have to seek proactively for new stakeholders. Face to face meeting with stakeholders is a good way to obtain and gather their concerns. A good way to obtain commitment is to actively engage your stakeholders. You need to build trust with your stakeholders. The question here will be who are the main stakeholders for the integration requirements? At the beginning of a project you as a project manager may not identified all the stakeholders of the project. As project manager you need to find an integration architect t in order to analyze your project from an integration point of view. For an integration architect it is important to understand the boundaries of the project and the processes, the information and as we mentioned previously the IT landscape that will be impacted by the project. When the impacted processes and information are identified then the architects can start with performing the map and creating the process/system and information/system matrixes and views. These deliverables are very important in order to: • Define the information to be exchanged and identify the information flows. • Define the security and other quality attributes requirements (e.g. guaranteed delivery, …) The stakeholders for the integration requirements will be the process and the information owners, security officers, IT (infrastructure, developers,…), the project manager and sponsor. As an integration architect or integration project manager (if that is needed as a sub- project) you may involve the users in order to check their view over the requirements and the desired outcomes of the integration solution. The list of the stakeholders could be much longer or shorter depending on the size and the impacts of the project. As an integration architect you need to gather and analyze the relevant requirements from the involved stakeholders. In many cases you may need to perform a pre-study to check the feasibility and the costs, impacts and the risks of these solutions. Nothing is new here, as requirement manager you need to identify, specify, analyze and agree about the use cases, the quality attributes and other requirements for your project. And as integration architect you need to translate these requirement (and maybe specify and detail more) these requirements into integration requirements.
  10. 10. 10 3.1.1 The requirements of a project The requirements on SAP integration or on a specific information flow that involves SAP, determine what SAP interface is the most suitable to fulfil the requirements. As an integration/solution architect or a developer with the responsibility to provide an accurate solution to the agreed requirements, you may need to have a holistic view over the requirements, especially if the project is very complex and involves many applications or system (e.g. implementing a new SAP ERP system and connecting the new system to internal and external systems). In this case and if you are the responsible for providing the integration solution, you will need to understand the processes and information needs behind these processes. Such kind of projects (e.g. integrating a new billing and CRM system to other internal and external systems) requires efforts from many business/IT architects, project managers, developers and others. As we mentioned in the previous section if the list of the use cases and the quality attributes are ready (gathered, specified, analysed and agreed) and the integration architect was involved in this process, then the translation to integration requirements will be easier than starting from nothing. The following is a list of required quality attributes that have the impacts in integration requirements and the integration solution: Quality Attributes Definition Reliability According to [23], reliability is the ability of a system to remain operational over time. Reliability is measured as the probability that a system will not fail to perform its intended functions over a specified time interval. Performance (response time) According to [23], performance is an indication of the responsiveness of a system to execute any action within a given time interval. It can be measured in terms of latency or throughput. Latency is the time taken to respond to any event. Throughput is the number of events that take place within a given amount of time. Using an integration platform like BizTalk Server and developing a messaging solution that uses the BizTalk message box will reduce the responsiveness of the system. Security Security is the capability of a system to prevent malicious or accidental actions outside of the designed usage, and to prevent disclosure or loss of information. A secure system aims to protect assets and prevent unauthorized modification of information, for more information, see [23]. In many cases the EAIP (Enterprise Application Integration Platform) is used as an interface to external systems. The main reason is that an EAIP system can handle the connectivity and receiving/sending messages in different formats with different
  11. 11. 11 protocols. Often these systems are designed with security in mind. Interoperability According to [23], interoperability is the ability of a system or different systems to operate successfully by communicating and exchanging information with other external systems written and run by external parties. An interoperable system makes it easier to exchange and reuse information internally as well as externally. In these guidelines and the cases that are presented In this document, we assume that these projects have interoperability requirements. Maintainability According to [23], maintainability is the ability of the system to undergo changes with a degree of ease. These changes could impact components, services, features, and interfaces when adding or changing the functionality, fixing errors, and meeting new business requirements. It is important to analyze if the sender and the receiver systems have integration capabilities out the box. Otherwise developing integration capabilities in the sender and the receiver systems could make these systems very complex and hard to maintain. Even complex Integration flows developed in an EAIP are easy to maintain and maybe reuse. Reusability According to [23], reusability defines the capability for components and subsystems to be suitable for use in other applications and in other scenarios. Reusability minimizes the duplication of components and also the implementation time. Creating Canonical Data Models helps organisations to reuse these models, reduce the amount of mappings and to have a common understanding of the data. Integration Capabilities Message Traceability, Message Archiving Message traceability is an out of the box capability of an EAIP and it enables tracing of metadata for messages sent through the integration platform. Ability to archive full messages or searching for messages by content are also capabilities that some of these platforms have. Message Routing The Message Router pattern determines the recipient of the message based on a set of conditions. The Content-Based Router inspects the content of a message and routes it to another channel based on the content of the message. Using such a router enables the message producer to send messages to a single channel and leave it to the Content-Based Router to inspect messages and route them to the proper destination. This alleviates the sending application from this task and avoids coupling the message producer to specific destination channels. Basic Message Router uses fixed rules to determine the destination of an incoming message. Where we need more
  12. 12. 12 flexibility, a Dynamic Router can be very useful. This router allows the routing logic to be modified by sending control messages to a designated control port. The dynamic nature of the Dynamic Router can be combined with most forms of the Message Router, for more information, see [4]. This integration capability can be difficult to develop in a sender system (we assume that the sender system is not an EAIP). If the requirements are indicating that a there are needs to route messages to different systems, it is wise to use an EAIP. Workflow and orchestration capability Orchestration or workflow capability enables multiple information flows to work as one where the process is coordinated within the integration platform. In common cases this is used for workflow logic i.e. which applications to call in which order and with what messages, but in more specific cases it can also be extended with business logic i.e. when the required business logic is not present in any one of the integrated applications. As the previous capability, this capability is an EAIP capability. Transformation capability In many cases, enterprise integration solutions receive, send and route messages between existing applications such as legacy systems, packaged applications, homegrown custom applications, or applications operated by external partners. Each of these applications is usually built around a proprietary data model. Each application may have a slightly different notion of the Customer entity, the attributes that define a Customer and which other entities a Customer is related to. For example, the accounting system may be more interested in the customer's tax payer ID numbers while the customer-relationship management (CRM) system stores phone numbers and addresses. The application’s underlying data model usually drives the design of the physical database schema, an interface file format or a programming interface (API) -- those entities that an integration solution has to interface with. As a result, the applications expect to receive messages that mimic the application's internal data format, for more information, see [4]. Transformation is one of the main capabilities of a EAIP. Table 1: quality attributes that have the impacts in integration solution
  13. 13. 13 Interoperability is one of the main required capabilities in every enterprise. Projects with IT components (e.g. purchasing a Supplier Relationship Management system, a Work Order Management system etc.) need to define and analyze the impacts of the current and future IT landscape on the purchased system and the impacts of the purchased system on the current and future IT landscape. This is important to align the purchased systems to the target architecture of the enterprise. When a project with interoperability requirements starts at some part of your organization then there will be needs to gather and analyze the integration requirements. Regardless of the setup of your organization, as a project manager you will need to appoint an integration architect/expert to specify the integration requirements on the project’s integration solution. As we mentioned at the beginning of this document, the integration team or an integration architect should be involved at the beginning of a project. Below is a basic questionnaire for any project that is about to integrate any two or more applications/systems. The questionnaire defines the general requirements on the information flow. The integration team asks (sends) these questions when they receives a request for integration from a project [1]. The following table lists the relevant questions that should be asked by the project to our stakeholders. The related requirements column will be very useful for the project to design and determine the right interface (see the matrix on page 19). You may need to add new questions and add new column “Stakeholder” and direct every question to a specific stakeholder or a group of stakeholders. Question Requirement Is it necessary to have ordered delivery of messages? Ordered delivery How critical is the information and what are the consequences of losing or delaying messages? (Cost, quality, reputation etc) Guaranteed Delivery Do the process and the information flow involves many systems? Interoperability, Workflow and Transactionality Are there any requirements for rolling back transactions when a part of transaction scope fails? Workflow and Transactionality Does the information have to have a request/response pattern or any other specialized communication pattern (e.g. synchronous / asynchronous, publish/subscribe, etc.)? Synchronous / Asynchronous communication What latency (delay) can be accepted for the delivery of messages? Performance Are there any special security requirements (on authentication, integrity,…)? - This involves mechanisms like encryption, acknowledges, etc. which all have an impact on cost and the complexity of the integration flow. Security
  14. 14. 14 How much data will be transferred per time (day, week and month) in kilobytes and in business related numbers/transactions (# of invoices, orders, meter readings, etc.)? Performance, large amount of data (data volume) Table 2: Relevant questions that should be asked to the integration solution stakeholders The answers to the above table’s questions define which requirements on the information flow are critical and should be considered by the project. The following requirements are the most important requirements when determining the right interface: o Transactional communication o Guaranteed Delivery o Performance (low latency) o In order Delivery 3.2 Definitions, Acronyms and Abbreviations We start with defining the main terms that we will use in the rest of this document. If you are familiar with these SAP terms and the other terms that are listed and defined in the next pages, you may jump to the next section of this document. 3.2.1 SAP related Definitions 3.2.1.1 RFC Remote Function Call is the technology that allows calling a function module within a SAP system from another SAP instance or from an external program.This method is used for event driven Architecture and for Synchronous Communication. The calling system is the RFC client; the system called is the RFC server. RFC is based on the RPC (remote procedure call) model of the UNIX TCP/IP environment. RFCs manage the communication process, parameter transfer, and error handling. Before an RFC model can be called from an SAP system, the import/export parameters must be known, and a technical connection must be exist. This is known as an RFC connection or RFC destination. For more information, see [13]. The following figure illustrates the types of RFCs that are explained in the next pages:
  15. 15. 15 RFC sRFC tRFC qRFC BAPI Figure 2: The RFC types Synchronous RFC The first version of RFC is synchronous RFC (sRFC). This type of RFC executes the function call based on synchronous communication, meaning that the systems involved must both be available at the time the call is made. Transactional RFC (tRFC) Transactional RFC (tRFC, previously known as asynchronous RFC) is an asynchronous communication method that executes the called function module just once in the RFC server. The remote system need not be available at the time when the RFC client program is executing a tRFC. The tRFC component stores the called RFC function, together with the corresponding data, in the SAP database under a unique transaction ID (TID). If a call is sent, and the receiving system is down, the call remains in the local queue. The calling dialog program can proceed without waiting to see whether the remote call was successful. If the receiving system does not become active within a certain amount of time, the call is scheduled to run in batch. tRFC is always used if a function is executed as a Logical Unit of Work (LUW). Within a LUW, all calls o Are executed in the order in which they are called o Are executed in the same program context in the target system o Run as a single transaction: they are either committed or rolled back as a unit. Implementation of tRFC is recommended if you want to maintain the transactional sequence of the calls.
  16. 16. 16 o Disadvantages of tRFC o tRFC processes all LUWs independently of one another. Due to the amount of activated tRFC processes, this procedure can reduce performance significantly in both the send and the target systems. o In addition, the sequence of LUWs defined in the application cannot be kept. It is therefore impossible to guarantee that the transactions will be executed in the sequence dictated by the application. The only thing that can be guaranteed is that all LUWs are transferred sooner or later. Queued RFC (qRFC) To guarantee that multiple LUWs are processed in the order specified by the application, tRFC can be serialized using queues (inbound and outbound queues). This type of RFC is called queued RFC (qRFC). qRFC is therefore an extension of tRFC. It transfers an LUW (transaction) only if it has no predecessors (based on the sequence defined in different application programs) in the participating queues. Implementation of qRFC is recommended if you want to guarantee that several transactions are processed in a predefined order. For more information, see SAP Help portal: http://help.sap.com/saphelp_nw70ehp2/helpdata/en/6f/1bd5b6a85b11d6b285005 08b5d5211/content.htm?frameset=/en/22/042671488911d189490000e829fbbd/fra meset.htm 3.2.1.2 BAPI BAPI function modules are special function modules that are released by SAP as an official application-programming interface, and they are supported for public use. Hence, BAPIs are a subset of the RFC enabled function modules. BAPIs are implemented and stored as RFC-enabled function modules in the ABAP Workbench and can also be used as a basis for WebServices. This function is used for Synchronous Communication. BAPIs play an important role in technical integration and business data exchange between SAP components and external components. For more information, see [13]. • Advantages o BAPIs can tell if the sending was successful or not o Sending to/processing on the other side is immediate o Easier to create custom BAPI than custom IDoc • Disadvantages
  17. 17. 17 o Will only work if there is an active online connection. o Some programming required to call a BAPI. We need to explain the differences between BAPIs and RFCs. This is very important due to the availability of the RFCs and BAPIs to be accessed by a non-SAP system. BAPIs and especially standard BAPIs (not customized) are by default available to be accessed (called) by a non-SAP system. RFCs by default are only available to be accessed by a SAP system. Technically, there is no difference between RFCs and BAPIs, but SAP wants you to use a BAPI to access R/3 functionality from external programs, whenever a BAPI exists [20]. It is not possible to connect SAP to Non-SAP systems to retrieve data using RFC alone. RFCs can access the SAP from outside only through BAPI and same is for vice versa access, for more information see [19]. 3.2.1.3 IDoc Intermediate Documents are a transport vehicle for data transfer in and out of an SAP system. The data transferred between the systems is in the IDoc format, which is a text format. This method is used for: • Event driven Architecture and for asynchronous Communication. • Sharing information by messaging Different message types (such as delivery notes and purchase orders) normally represent the different specific formats, known as IDoc types. Business data is exchanged between systems by means of electronic data interchange (EDI) and application link enabling (ALE). The IDoc interface, in which a data structure and corresponding processing logic is defined, is required for both forms of data transfer. The structure of the IDoc is described by the IDoc type. This contains, for example, information about where data is stored. For more information, see [13]. Benefits of IDocs are: they are event driven, they are readable in the case of a malfunction, they ensure transactionality and traceability and they offer a future- proof solution, especially if the destination system wants to receive the information indirectly. It should be taken into consideration that to make system and IDoc processing optimal, IDocs should be processed in the background, otherwise the system can be blocked by too many IDocs all using dialog processes. • Advantages o System can work even if target system not always online. The IDoc will be created and sending will just continue once you get connected to the other system.
  18. 18. 18 o No additional programming required. You just need to set up the configuration. • Disadvantages o Receipt/processing on the target system may not be immediate. o The sending system has no way to know whether the target system actually received what you sent (unless you use ALE). o If there is no SAP standard IDoc available for your requirement, it's harder to create a customized IDoc than a customized BAPI. o The output of an IDoc structure schema “the flat-file” will be filled with zeros for empty positions in a fileds and the rest of a partly filled field, and therefore IDocs can be very bandwidth-intensive. 3.2.1.4 ALE ALE technology: Application Link Enabling (ALE) is a technology to create and run distributed applications. ALE Objectives - ALE incorporates controlled exchange of data messages ensuring data consistency across loosely coupled applications. Basic principle of ALE is to provide a distributed and fully integrated R/3 system. Difference between ALE and EDI: Normally we refer to EDI technology when a non- SAP system is one of the communication channel. ALE communication occurs from the SAP side and EDI from the non-SAP side. IDocs use ALE and EDI to deliver the data to the receiving system. If the data needs to be exchanged between two SAP systems, then IDOC uses ALE technology. For the exchange of data between a SAP and Non SAP system, IDOC uses EDI subsystem to convert and deliver the data, see [17]. ALE is a slightly older but still valid means of creating and operating distributed applications involving the monitored exchange of consistent data between systems. ALE consists of application services, distribution services, and communication services. Both master data and application data can be exchanged using ALE. For more information, see [13]. • Advantages o ALE offers better inbound interface performance compared to traditional techniques such as Batch Data Communications (BDC) or call Transactions. ALE does not use screen-based batch input. o ALE is provides “loose coupling” with legacy and third-party applications and is a Business Framework key element. It provides a message-based architecture for asynchronous integration of Business Framework components, including Business Components, Business Objects and BAPIs, for more information, see [15].
  19. 19. 19 • Disadvantages o The disadvantage of asynchronous communications via ALE is that only a single return parameter from the called system is available, see [15]. 3.2.2 Other Definitions Many of the following definitions are based on the material provided in the following webpage [4]: http://www.enterpriseintegrationpatterns.com/ In our integration framework we used the patterns provided by Gregor Hohpe and others in many of our integration solutions, see the reference [2]. 3.2.2.1 Adapters and Interfaces Adapters (Channel Adapters): Many enterprises use Messaging to integrate multiple, disparate applications. How can you connect an application to the messaging system so that it can send and receive messages? Figure 3: Adapters (Source [4]) Use a Channel Adapter that can access the application's API or data and publish messages on a channel based on this data, and that likewise can receive messages and invoke functionality inside the application. The adapter acts as a messaging client to the messaging system and invokes applications functions via an application-supplied interface. This way, any application can connect to the messaging system and be integrated with other applications as long as it has a proper Channel Adapter. Interface or Messaging Bridge: An enterprise is using Messaging to enable applications to communicate. However, the enterprise uses more than one messaging system, which confuses the issue of which messaging system an application should connect to. How can multiple messaging systems be connected so that messages available on one are also available on the others?
  20. 20. 20 Figure 4: Interface/Message Bridge (Source [4]) Use a Messaging Bridge, a connection between messaging systems, to replicate messages between systems. Typically, there is no practical way to connect two complete messaging systems, so instead we connect individual, corresponding channels between the messaging systems. The Messaging Bridge is a set of Channel Adapters, where the non- messaging client is actually another messaging system, and where each pair of adapters connects a pair of corresponding channels. The bridge acts as map from one set of channels to the other, and also transforms the message format of one system to the other. The connected channels may be used to transmit messages between traditional clients of the messaging system, or strictly for messages intended for other messaging systems. 3.2.2.2 Guaranteed Delivery Distributed systems over many networks, by default, may not guaranty the delivery of messages between the involved systems. Guaranteed Delivery is a mechanism that involves more or less complex behavior and measures (e.g. storing, acknowledges, using reliable protocols etc) from the sender and the receiver systems. According to [4], if the assumption is that the enterprise is using Messaging to integrate applications. Then the question is how can the sender make sure that a message will be delivered, even if the messaging system fails? With Guaranteed Delivery, the messaging system uses a built-in data store, which enables messages to persist. Each computer the messaging system is installed on, has its own data store so that the messages can be stored locally. When the sender sends a message, the send operation does not complete successfully until the message is safely stored in the sender’s data store. Subsequently, the message is not deleted from one data store until it is successfully forwarded to and stored in the next data store. In this way, when the sender successfully sends the message, it is always stored on disk on at least one computer, see the figure below, until it is successfully delivered and acknowledged by the receiver. For more information, see [4].
  21. 21. 21 Figure 5: Guaranteed Delivery (Source [4]) 3.2.2.3 Transactional messages A messaging system, by necessity, uses transactional behavior internally. It may be useful for an external client to be able to control the scope of the transactions that impact its behavior. The question is: How can a client control its transactions with the messaging system? Both a sender and a receiver can be transactional. With a sender, the message isn’t “really” added to the channel until the sender commits the transaction. With a receiver, the message isn’t “really” removed from the channel until the receiver commits the transaction, see the figure below. A sender that uses explicit transactions can be used with a receiver that uses implicit transactions, and vise versa. A single channel might have a combination of implicitly and explicitly transactional senders; it could also have a combination of receivers. For more information, see [4]. Figure 6: Transactional Messaging (Source [4]) When a sender and/or a receiver use files to integrate their application, they cannot guarantee the delivery, because files are not transactional. Transactional behaviour involves complex mechanisms like commit or rollback features.
  22. 22. 22 To explain more about transactional behaviour, we take another example from the SOA patterns. According to [22], as per the next figure, Services A and B complete their respective tasks successfully. However each time they do, they initiate a local transaction, temporarily saving the current state of the database prior to making their changes (1, 2). After Service C fails its database update attempt (3), Services A and B restore their databases back to their original states (4, 5). The business task is effectively reset or rolled back across services within the pre-defined transaction boundary. Figure 7: Transactional Messaging (SOA) (Source [22]) 3.2.2.4 Event driven messaging An application needs to consume Messages as soon as they’re delivered. Here the question is: How can an application automatically consume messages as they become available? The application should use an Event-Driven Consumer, one that is automatically handed messages as they’re delivered on the channel, see the figure below. This is also known as an asynchronous receiver, because the receiver does not have a running thread until a callback thread delivers a message. For more information, see [4].
  23. 23. 23 Figure 7: Event Driven Messaging (Source [4]) 3.2.2.5 Synchronous and Asynchronous Communication Communication between two systems can be basically split into two types: Synchronous and asynchronous communication. Both forms of communication have specific advantages and disadvantages, relating to either the business application or the system administration. 3.2.2.6 Synchronous communication According to [25], synchronous communication uses a single function call. A prerequisite for this is that at the time the call is made (or the message is sent), the receiving system is also active and can accept the call and further process it if necessary, see the next figure. Advantage: Synchronous communication can be implemented in function calls that require the immediate return of data to the sender system. Example: You create a purchase order with account assignment in the sender system, and you want to perform a budget check in central accounting before you save the purchase order. Another example is when the customer call center checks the credit of a new customer. Disadvantage: You need to ensure that both systems are active and can be contacted. If they are not, this can lead to a serious disruption of processes. In particular, problems can arise if the receiving system is not available for long periods of time due to maintenance (for example, for a system upgrade) [25]. Figure 8: synchronous communication(Source [25])
  24. 24. 24 3.2.2.7 Asynchronous Communication According to [25]: For asynchronous communication, the receiving system does not necessarily have to be available at the time a function call is dispatched from the sender system. The receiving system can receive and process the call at a later time. If the receiving system is not available, the function call remains in the outbound queue of the sending system, from where the call is repeated at regular intervals until it can be processed by the receiving system, see the next figure. Advantages: The receiving system does not have to be available at the time the function call is made. If the system is unavailable for a long period of time, for example, for an upgrade, it can still process the data that has been sent in the interim at a later time, and processes in the sending system remain unharmed. Example: You are sending a purchase order to a vendor system. The sending system cannot influence the availability of the receiving system. If the receiving system is not available, the purchase order can be sent repeatedly until the vendor system is available again. In asynchronous communication, you usually have the option to send data (for example, business documents or changes to master data) in packages or individually (immediately). Note that the Send Immediately option in asynchronous communication should not be confused with the method synchronous communication). The advantage of sending data in packages is that system resources are employed more efficiently, because each function call occupies one work process in the system. Example: You want to distribute 100 material master changes to other systems. If you send the changes in a package (with 100 pieces) you only require one work process. If you sent the same 100 material master changes individually, you would need 100 work processes in the system. When using asynchronous communication, you should therefore always carefully consider the availability of your system resources and the necessity of immediate data transfer. Disadvantage: Processes that require an immediate response to the sender system cannot be executed using this method.
  25. 25. 25 Figure 9: Asynchronous communication n(Source [25]) 3.2.2.8 WebServices (with Reliable Messaging) According to [26]: Whenever you are using Web services in business-critical applications, it is important that messages are exchanged in a reliable manner. Web Services Reliable Messaging (WSRM) ensures that message exchange is performed correctly – without messages getting lost or being duplicated. WSRM ensures a reliable exchange of messages even when the connection to the network is lost – for example, during a purchase order. To guarantee message exchange and also to control the sequence of incoming messages, the WSRM protocol combines one or multiple messages into sequences. Sequences contain a unique identifier. Messages in a sequence are numbered consecutively. The WSRM sequence header in the SOAP message identifies the sequence to which a message belongs. WS Reliable Messaging implementations at the sender and receiver sides ensure that messages are transferred securely. A prerequisite for this is that incoming messages are confirmed by the receiver. For this purpose, the specification defines the format of an acknowledgement that the receiver sends to the sender as confirmation. The sender waits for the confirmation and, if necessary, keeps sending the message until the confirmation is received.
  26. 26. 26 3.2.2.9 WebServices (without Reliable Messaging) You can either use the inside-out or outside-in approach to develop a point-to-point communication using Web Services (more information, see [27]). Both approaches work with the WSDL standard (WSDL: Web Service Description Language) to transfer all the necessary information for calling the service to the service consumer. The development of point-to-point communication using classic Web Services is performed using the inside-out approach: a WSDL document, in which all the necessary information for calling the function is contained, is generated for a pre- existing function in an application system. Using this WSDL document, consumers can generate a runtime substitute (a proxy) on their platform and in this way the consumer application can call the provider’s service. In outside-in development, you work with service interfaces (outbound and inbound), which you create directly in the ES (Enterprise Services) Repository. Service interfaces are language-independent and are based on the WSDL standard. Using this description, you can generate language-specific proxies, which you can then use to implement the actual message exchange. The generated proxies are part of the application and forward calls to the Web Service runtime that executes the point-to-point message exchange. For more information, see [10]. 3.2.2.10 SFTP In computing, the SSH File Transfer Protocol (sometimes called Secure File Transfer Protocol or SFTP) is a network protocol that provides file access, file transfer, and file management functionality over any reliable data stream. It was designed by the Internet Engineering Task Force (IETF) as an extension of the Secure Shell protocol (SSH) version 2.0 to provide secure file transfer capability, but is also intended to be usable with other protocols. The IETF of the Internet Draft states that even though this protocol is described in the context of the SSH-2 protocol, this protocol is general and independent of the rest of the SSH2 protocol suite. It could be used in a number of different applications, such as secure file transfer over Transport Layer Security (TLS) and transfer of management information in VPN applications. This protocol assumes that it is run over a secure channel, such as SSH, that the server has already authenticated the client, and that the identity of the client user is available to the protocol [7]. SFTP is a viable solution in order to comply with the security issue. For more information about File, see the next section. 3.2.2.11 Files Using files as the transport in integration is by far the most common way to convey information between information systems. Files are great for storing data but as a means for secure integration, they are not the best alternative. Consider the following when using files as transport in integrations: Advantages: Files are easy to apprehend; everyone understands the concept of a file.
  27. 27. 27 The tools to manipulate files (for viewing, deleting, appending, etc) are known by everyone and built into the operating system. Disadvantage: Files are NOT transactional, that is; there is no standard way to ensure that ‘all or nothing’ is transferred. A common error in all file-based integration is that if a file transfer is disrupted, a fraction of the file is transferred and the rest is not. In order assure transactionality when using files, additional mechanisms must be added on top, i.e. create a proprietary protocol. This makes it complicated and thereby expensive and error prone if good transactionality is required. A possible solution would be to add control sums to files manually or by means of existing solutions for example in archiving programs (gzip). Files names are the only way to separate one file from another. This makes file naming sensitive and one must be extra careful to ensure unique file names. A common mistake is to use hours, minutes and seconds of the time stamp as part of the file name. If more than one file is received in the same second, files names will not be unique and there will be a collision error. File names should ensure the unique identification of the file; therefore using the time stamp should be avoided. Use sequence number or a randomized GUID (globally unique identifier) if possible. As a result of the facts above, it is not uncommon to add functionality to the basic file transport to ensure transactionality and the unique identity of messages being conveyed. This is close to designing your own proprietary protocols, which will certainly drive up development costs. If transactionality and unique identity are important characteristics, file transport should be avoided. SFTP could be a viable solution to comply with the security issue. 4. Plan Now we defined the important components of an integration flow that involves one or more SAP systems and we know more about the integration requirements. In order to design an accurate solution and plan the implementation of the integration solutions, we need to: 1. Define the connectivity possibilities for the involved applications 2. Define the integration requirements 3. Define the solution or the alternative solutions 4. Estimate the required time to implement the solution 5. Create a budget 6. Plan the implementation
  28. 28. 28 4.1.1 Connectivity to applications Another important parameter that should be taken into consideration is the connectivity possibilities of the application that needs to be integrated with SAP as well as the connectivity possibilities of the SAP application. The following questions should be asked by the project to the system manager of the applications to be integrated with SAP and to the system manager of the SAP application: • How can data be accessed in the application(s)? • Are there any proprietary connectivity mechanisms, such as APIs, SAP IDocs, BAPI, etc? • What is the geographical location of the application hardware? • Are there any other considerations that must be handled regarding firewalls and networking, like applications being placed outside the organization’s internal network, behind its own firewall or in a DMZ, etc. (Note this must be known for all environments, not only for production instances of the integrated applications). The above questionnaire defines the general characteristics of the application that needs to be integrated with SAP. 4.1.1.1 How SAP communicates/interfaces with other systems? As we mentioned previously, in order to support business processes and data sharing across applications, applications need to be integrated. Application integration needs to provide efficient, reliable, and secure data exchange between multiple enterprise applications [2]. Applications need to send and receive information from other applications to complete their process. The information could be shared by: • Sending/receiving files • Sharing databases; this type of sharing of information is not included in the scope of this document • Remote procedure invocation • Messaging As we mentioned earlier, SAP uses BAPI/RFC, IDocs, ALE, WebServices with Reliable Messaging, WebServices without Reliable Messaging and Files to receive and send messages from/to different destinations.
  29. 29. 29 The diagram in the following figure, explains how the interfacing mechanisms in SAP is working. The diagram lists the main adapters that are available in the SAP environment. Some of these adapters are available via the SAP R/3 and mySAP ERP and other adapters are available via the SAP NetWeaver Application Server and SAP NetWeaver Process Integration. There are of course many other standard and third party adapters that are not listed in the following figure. For example, from the company SEEBURGER all adapters that are covered by the reseller agreement between SAP and SEEBURGER are already certified for SAP NetWeaver PI 7.1; to be more specific, the following SEEBURGER adapters for SAP NetWeaver PI 7.1 that are covered by the reseller agreement are available: (see, [21]) Technical EDI Adapters AS2 (EDIINT/HTTP(S)) OFTP (OFTP / ISDN, OFTP / TCPIP) VAN Access (P7 / X.400, VAN FTP) We will not define or use these adapters, because they are not included in the scope of the guidelines.
  30. 30. 30 SAP NetWeaver Application Server Web Service SAP R/3, mySAP ERP IDoc Inbound/ Outbound Processing BAPI/ RFC ALE Inbound/ Outbound Processing File ….. …... Figure 10: SAP Interface architecture 4.1.2 The decision matrix As we mentioned in the beginning of this document, the alternatives for connecting two or more system that have different possibilities to be accessed could be more than one. If we analyze the following figure we see that in the left side we have the requirements and in the right side we have the interfaces. Our assignment is to select the right interface for the agreed requirements.
  31. 31. 31 Requirements Security Performance Interface Maintaina- bility Reusability Common Quality Attributes Specific Integration Requirements Guaranteed Delivery Transactionality Ordered delivery WebService ALE BAPI/RFC IDoc File/(S)FTP …. Synchronous communication Message Routing, Traceability, Orchestration,... Asynchronous communication Figure 11: Integration Requirements vs Interfaces If we assume that we can use asynchronous communication for our messages, then the alternative interfaces will be: Asynchronous communication WebService ALE BAPI/RFC IDoc File/FTP SFTP Figure 12: Many Alternative Interfaces for a Requirement The following table describes the different SAP interfaces and the corresponding requirements that they implement. It should be helpful when making the decision about which interface is most suitable and meets the requirements:
  32. 32. 32 Table 3: Decision Matrix Explanations: Async. Com.: Asynchronous communication; No = not allowed; Yes = shall be used; Sync. Com.: Synchronous communication; A = Applicable: There are possibilities to fulfill the requirement (depending on the integration scenario and on the provided/available interfaces); N/A = Not Applicable (the interface has not the technical capabilities to fulfill the requirements); 2 With HTTPS; 3 With HTTPS; 4 If there are no requirements on Transactionality; Transactionality: In this scope we define Transactionality as ‘the ability to ensure that a message is conveyed in its entirety’. The whole message or nothing at all is sent. 5 By using a program 6 If there are no requirements on Transactionality 7 By using a program Interface Sync. Com. Async. Com. Guaranteed Delivery Reusable Large amount of data Scheduling mechanism Transactionality Security IDoc (Inbound & Outbound) N/A Yes Yes Yes Yes Yes Yes Yes ALE N/A Yes Yes Yes Yes Yes Yes Yes RFC/BAPI Yes A Yes Yes No N/A Yes Yes Web Services with RM Yes A Yes Yes No N/A Yes Yes2 Web Services without RM Yes A No Yes No N/A No Yes3 File (FTP) N/A Yes4 No No Yes Yes5 No No SFTP N/A Yes 6 No No Yes Yes7 No Yes
  33. 33. 33 4.1.3 Determining which interface method to use when interfacing to SAP Depending on the requirements on the SAP interfaces (e.g. Guaranteed Delivery, Transactional messages , the reliability and performance of the interface), there are several ways to communicate with SAP (see the table) and there are at least two cases to handle: Case 1: If the requirements include Guaranteed Delivery, Transactional messages, event driven messaging: The projects have to use suitable and transactional/reliable SAP interface technologies like: BAPI/RFC, IDocs, ALE, WebServices with Reliable Messaging (if the SAP version provides WebServices or supports the services paradigm) to receive and send messages from/to different destinations. Case 2: If the risk analysis assesses that the consequences of losing messages are not serious for the business (there areno needs for Guaranteed Delivery, Transactional messaging): • It is permissible to use File integration (P2P non-transactional integration). SFTP is a viable solution in order to comply with the security issue. • It is permissible to use WebServices (without Reliable Messaging). 4.1.3.1 Integration Scenario To create a total list of integration scenarios that are possible when integrating applications with many adapters like SAP, requires great effort. The list depends on: • The requirements on the interfaces • SAP to SAP scenarios • SAP to non-SAP scenarios • Internal/External applications that want to communicate with SAP • The available adapters that exists in the sending/receiving applications • As mentioned earlier, the availability of an EAI tool in the organization (e.g. BizTalk Server, SAP PI/XI and the existence of SAP adapters to handle IDocs, BAPI, RFC messages). • The volume of the messages is a very important parameter in defining a reliable interface. You need to consider an interface that is using a BAPI is not suitable for large messages. It is wise to run performance tests for the designed interfaces. The figure below lists some of the common scenarios: 1. Scenario 1: an application has the possibility to connect to a WebService to receive information by http/https protocol.
  34. 34. 34 2. Scenario 2: an application has the possibility to send IDocs to SAP. 3. Scenario 3: an application has the possibility to receive IDocs from SAP. 4. Scenario 4: an application has the possibility to connect to a BAPI/RFC. 5. Scenario 5: an application has the possibility to receive a File from SAP. 6. Scenario 6: an application has the possibility to send a File to SAP. 7. Scenario 7: an application has the possibility to receive ALE from SAP. 8. Scenario 8: an application has the possibility to send ALE to SAP. SAP NetWeaver Application Server Web Service SAP R/3, mySAP ERP IDoc Inbound/ Outbound Processing BAPI/ RFC ALE External/ Internal Systems …. SAP SAP SAP ….File Scenario 4 Figure 13: Integration scenarios 4.1.4 Estimate the solution As main principle, the estimation, requirements specification (for the integration part) and the design of the integration solution will be the responsibility of the project. If we assume that the enterprise invested in an EAIP (an EAIP is available for the enterprise). And if we assume that the organization has an integration team or (integration competence center), then it is natural that estimation of the required efforts for building the integration solution and the required integration flows will be provided by the experts from the integration team.
  35. 35. 35 If assume that the enterprise uses the most popular application integration tool (BizTalk Server 20XX), the BizTalk Architects and the BizTalk developers needs to provide a diagram for each integration flow. The following diagram is taken from a Microsoft page [28]. The diagram illustrates the flow of a message through the BizTalk Server. The main steps in the message flow are: 1. A message is received through a receive port which contains: Receive Adapter, Receive Pipeline and Inbound Map. The received massage from the sending system will be handled by a pre-configured adapter. BizTalk has many adapters (e.g. WCF-SAP, HTTP, FTP, File, etc). 2. The receive pipeline processes the message and perform different operations (e.g. decoding, disassembling, validating, etc). 3. The receive ports may transform a message to an internal message format defined and deployed in the BizTalk environment. 4. The message is inserted into the MessageBox which is based on a SQL Server database. And according to the publish-subscribe pattern the subscribers (e.g. Orchestration or Send Ports) will receive a copy of the right message that the subscriber is listening to. 5. If the subscriber is an orchestration then the orchestration will process the message and executes the required workflow and sends the message back to the MessageBox. 6. If the subscriber is a send port, then the message can be transformed into a required output format. 7. Before the message will be send to the destination, the send pipeline perform some operations like pre-Assembling, assembling and encoding. 8. The send port uses the pre-configured adapter to transmit the message to the destination system (e.g. SAP). The above steps and the requirements part of an integration solution will define the work breakdown structure (WBS) for an integration flow, see the next table.
  36. 36. 36 1 2 3 4 5 6 7 8 Figure 14: The path of a message in BizTalk Server The work breakdown structure (WBS) for an integration flow is divided into different sections. The main sections are: • Business Specification • Sending Application • Receiving Application • Information Flow A • General Activities Business Specification Comments Process description The integration team may request if there are any Process descriptions available for the project. Integration scenario Describe the Integration Scenarios: The type of integration tool to be decided (integration platform or point-to-point). Specification of use cases Analyse the provided use cases from the project and check the impacts of the use cases on the Integration solution. Create a use case diagram (for the integration requirements if needed). Specification of the flow characteristics The SLA parameters for the integration flow (or a group of integration flows) need to be agreed with the business/project. Sending Application Repeated for each sending application that is included in the project. General knowledge of application Understanding the main capabilities provided by the application. Analyse the application architecture and the underlying technology and building blocks of the application.
  37. 37. 37 Provide documentation and contact Find the contact persons (system owner, system manager) for the application. Connectivity at Sending Application Clarify the options available Selection mechanism for connecting Includes choice of standard adapters, custom designed adapters Design or configuration and testing of the adapter Environments Setting up test environment for sending application Server Configurations Receiving Application Repeated for any receiver application that is included in the project General knowledge of application See above for Sending Application Provide documentation and contact See above for Sending Application Connectivity at Receiving Application See above for Sending Application Clarify the options available See above for Sending Application Selection mechanism for connecting See above for Sending Application Environment See above for Sending Application Setting up test environment for receiver application See above for Sending Application Information Flow A Repeated for each information flow Data mapping Writing specification Data Mapping Specification document Workshops, meeting with users Design of the mapping Testing the mapping (unit test) Integration Logic Realized often through orchestration in BizTalk (if needed) Design Design of the orchestration Testing the integration logic (unit test) Quality assurance Specification of test data Constructing test data Generate test data Providing test data System Test Configuration of test generators Deploy to Acceptance Test environment Acceptance test Deploy Deploy to Production environment Initial Load Activation of service monitoring Supervision of queues, info flow measurement Handover of Operation Update on the operation documentation Operating instructions, interface contracts Archiving of project documentation Handover Meeting Approval of deliverables General Activities Project Management Table 4: WBS for an Information Flow
  38. 38. 38 4.1.5 Create a budget The budget will be based on the estimation of all required information flow from the project. As a project manager and with agreement with the integration Architect and the integration developers you may add 5-10% of the total time of the estimated hours/days for the integration flows. You may use that extra time in case something goes wrong. 4.1.6 Plan the implementation In my experience, the best way to define and agree about the integration requirements is to conduct workshops with the stakeholders. It is very important to prepare to these workshops (e.g. reading the available documents for the project, sending questions to different stakeholders, understanding the main concerns and the ambition levels of the stakeholders, preparing the main alternatives “if possible”). As an integration architect you need to involve the developers in an early stage. This is important to make the developer engaged and to see the solution from a developments point of view. As an integration architect you may miss some hidden sides or did not analyze the complexity of the integration flows. Planning the implementation is the task of the project manager and the team manager (line manager of the integration team).
  39. 39. 39 4.2 Do 4.2.1 A Case Study (how to use this document) In general, it is a big challenge to determine a right interface. The integration architects and the developers from the application teams should have enough knowledge not only about their applications but also regarding the interfacing requirements. Today, the best way to get an integration project successfully implemented is to use the expertise of an experienced team who did the job in the past or has the competence and the theoretical basis to do the design and implement an accurate solution. The Case Study: Imagine that a Work Order Management System (WOMS) needs to be updated with different information from different systems. For simplicity we will concentrate on the required information from HR (human resources). The data is in the SAP ERP system. The WOMS need to be updated when a new employee in the Service Organisation is hired. You as an integration architect/expert will be involved the Work Order Management System project. 1. Before analyzing the requirements, we need to identify the stakeholders of the integration requirements/the integration solution. As an integration architect/expert you may start with the project manager. With help of the project manager, you may start to identify the main stakeholders: (e.g. the HR process/information owner, HR administrator, the WOM process/information owner, developers, system manager of the involved systems, other the stakeholders “WOMS Dispatcher, the field worker”). 2. As we mentioned previously, when the stakeholders are identified, the project should begin with gathering and analysing the requirements on the information flow(s). We need to discuss and send the questions to our stakeholders, see table 2; relevant questions that should be asked to the integration solution stakeholders. Our assumptions is: The information about the new employee should be available as soon as possible to the WOMS in order to assign new orders to the employee “the field worker”. The information should be sent/fetched directly (event driven pattern) and the information flow requires Guaranteed Delivery. HR data by the nature is sensitive and need to be securely transmitted between applications. 3. When the requirements are defined the available interfaces need to be analysed, then the project selects the right interface from the decision matrix, by matching the requirements to the right interface. a. The analysis for the connectivity possibilities for the application “WOMS” that needs to be integrated with SAP shows that the application:
  40. 40. 40 i. WebService Client/ WebService: Can connect to other systems with a WebService Client or provides WebService to exchange information by http/https protocol. ii. Can connect and be connected by File interface b. The analysis for the connectivity possibilities for the SAP application shows that the application: i. Has a standard IDoc and standard BAPI for processing HR data. ii. Can connect and be connected by File interface 4. When we map the requirements to the available ways to communicate between these two applications, you see (according to the decision matrix) that in principle, the project has four alternatives for integration with SAP and one of these alternatives is not recommended (using files when the project requires Guaranteed Delivery for the message coming from SAP ERP). a. Alternative 1: The first alternative is based on the possibility that the SAP system can connect to a WebServices. The SAP system can access the WOMS with WebServices client over the https protocol. An asynchronous WebServices client/WebServices intraction with Reliable Messaging is the best alternative here. The advantage with this solution is that the SAP system can send the information about the new employee as soon the information is available in the SAP system. Sending application SAP ERP Receiving application WOMS XML XML WebService WebService Client Figure 15: The SAP system can connect to a WebServices
  41. 41. 41 b. Alternative 2: The second alternative is based on the possibility that the SAP system has with providing WebServices. The WOMS can access the SAP system with WebServices client over the https protocol. An asynchronous or synchronous WebServices client/WebServices interaction with Reliable Messaging could be a good alternative here. The disadvantage with this solution is that WOMS need to ask/request data from the SAP system without knowing that a new employee is hired or not. You need to schedule this request at an optimal frequent in order to minimize the load and at the same time detect a new employee as soon as possible. Sending application SAP ERP Receiving application WOMS XML XML WebService WebService Client Figure 16: The SAP system can be connected with a WebServices c. Alternative 3: The alternative is based on exchanging information via files. The SAP system sends a file every week or day or when a new employee is arrived. This alternative is not recommended, but it could be used if you do not have the possibilities in alternative 1, 2 and 4, for more information about why This alternative is not recommended, see the sections about Files/FTP. Sending application SAP ERP Receiving application WOMS File File Figure 17: Exchanging information via files
  42. 42. 42 d. As we mentioned in the beginning, we assume that the enterprise has invested previously in an Integration platform (as we did in 2004 and our platform BizTalk Server 2013 Platform now is one of the biggest platforms in Europe). One of the biggest advantages with an EAIP (Enterprise Application Integration platform) is the out of the box features (the availability of the integration capabilities). Systems like SAP, WOMS, CRM, SRM, etc, do not need to implement these capabilities. This makes the live easy for these systems. Systems that need to be connected do not need to be more complex than they are. Adapting these systems to receive and send messages to different systems with different formats, protocols and connecting/implementing different adapters, takes a lot of efforts and makes the maintainability for these systems a time consuming task. The integration platform among other integration capabilities, receives, sends and routes the messages from the receiving and to the sending systems. An integration platform with many out of the box features (e.g. different adapters) has the ability to be configured and adapted to the interoperability possibilities of the sending and receiving systems. If we add the integration platform to the figure number 13: Integration scenarios, the scenarios will changed, see the following figure: SAP NetWeaver Application Server Web Service SAP R/3, mySAP ERP IDoc Inbound/ Outbound Processing BAPI/ RFC ALE External/ Internal Systems …. ….. …. SAP ….File Integration Platform …. …. …. …. Figure 18: Integration Scenarios with involvement of an Integration Platform
  43. 43. 43 The disadvantage with involving an EAIP is the performance (the delay when a message will pass another system “the EAIP” (adapters, database, etc) before receiving the message in the destination system, see figure 14: The path of a message in BizTalk Server. Another disadvantage is that a project needs to deal and work with an additional team (the integration team) when the project needs to define the requirements, design the solution and implement, test, handover the outcomes. This could take time especially if the integration team is loaded with many simultaneous assignments. An integration solution based on involvement of an EAIP (in our case BizTalk Server 2013) could look like the following picture, see figure number 19. The project should avoid a point-to-point solution without Transactionality and Guaranteed Delivery, because of the requirements. In this alternative we think that the right interface should be IDocs, because an IDoc is suitable for event-driven and transactional messages, see the decision matrix. When a new employee will be recruited to the company and he/she need to be added to the organisation that works with Work Orders, a HR administrator uses the SAP system to insert this new employee into the ERP system. A SAP ABAP programmer configures and allows that an IDoc will be trigged when changes accrue in HR administration (in our case new employee arrived in the organisation X). The IDoc will be received in the BizTalk platform by the WCF SAP adapter. The message will be mapped to the Work Order Management System’s WSDL WebService and will be sent to WOMS over the https protocol. Sending application SAP ERP Receiving application Enterprise Application Integration Platform SAP Adapter XMLIDoc WebService Adapter IDoc XML Figure 19: Integration Flow with involvement of an Integration Platform 5. Check and Act The scope of this document is not to implement the interface, test it and acting if we find errors. We may need to monitor the requirements and check the impacts of changed requirements and new requirements in the integration solution.
  44. 44. 44 As we mentioned in table 4: WBS for an Information Flow, modifications need to be identified analysed and estimated. 6. References Reference Address 1 Requirement Specification for Application integration, Åke Svilling, Alaa Karam 2 Enterprise Integration Patterns, Gregor Hohpe,…, ISBN 9780321200686 3 http://www.info-sun.com/docs/wp_sapinter.pdf 4 http://www.enterpriseintegrationpatterns.com/ 5 http://www.dataxstream.com/wp-content/uploads/2009/03/sap-interface-design.pdf 6 http://www.sdn.sap.com/irj/sdn/webservices#section2 7 http://en.wikipedia.org/wiki/SSH_File_Transfer_Protocol 8 The position of SAP (Tino Scholman) 9 http://specs.xmlsoap.org/ws/2005/02/rm/ws-reliablemessaging.pdf 10 http://help.sap.com/saphelp_nwesrce/helpdata/en/45/614268d3f83bdbe10000000a1553f7/content.htm 11 http://en.wikipedia.org/wiki/Event-driven_architecture 12 http://help.sap.com/saphelp_nw04/helpdata/en/47/8f18e9aeba11d6b29500508b6b8a93/content.htm 13 SAP for Retail, Heike Rawe, ISBN 978-1-159229-213-4 14 http://www.soapatterns.org/event_driven_messaging.php 15 http://help.sap.com/saphelp_nw04/helpdata/en/18/22b800773211d396b20004ac96334b/content.htm 16 http://help.sap.com/saphelp_nw04/helpdata/en/0b/2a6524507d11d18ee90000e8366fc2/content.htm 17 http://wiki.scn.sap.com/wiki/display/ABAP/ALE,IDOC 18 http://www.eaipatterns.com/RequestReply.html 19 http://www.saptechies.org/difference-between-rfc-bapi/ 20 http://searchsap.techtarget.com/answer/Determining-which-interface-method-to-use-when-interfacing- to-SAP-R3 21 http://scn.sap.com/people/udo.paltzer/blog/2008/10/31/sap-netweaver-process-integration-71-available- adapters-and-adapter-modules 22 http://soapatterns.org/design_patterns/reliable_messaging http://soapatterns.org/design_patterns/atomic_service_transaction 23 http://msdn.microsoft.com/en-us/library/ee658094.aspx 24 http://www.eaipatterns.com/MessageRoutingIntro.html 25 https://help.sap.com/saphelp_nw04/helpdata/en/47/8f18e9aeba11d6b29500508b6b8a93/content.htm 26 http://help.sap.com/saphelp_nw70/helpdata/en/46/9743916d1115ece10000000a114a6b/content.htm 27 http://help.sap.com/saphelp_nwesrce/helpdata/en/60/00623c4f69b712e10000000a114084/content.htm 28 http://social.technet.microsoft.com/wiki/contents/articles/666.recommendations-for-installing-sizing- deploying-and-maintaining-a-biztalk-server-solution.aspx 29 http://msdn.microsoft.com/en-us/library/aa561360.aspx
  45. 45. 45

×