WSO2Con2011: Using WSO2 ESB with SAP ERP (Retail)


Published on

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • has over 170,000 customers in more than 120 countries.
  • High-level of technology adoption –
  • .com
  • Made sense but IDOC parsing and transformations and handling reliable messaging would have been LOT of effort! Light weight, open source, met our key requirements, local support, SAP adapter was being developed, joint go-to-market strategy Claimed Ebay processes 1 billion transactions a day.. So why not?
  • Made sense but IDOC parsing and transformations and handling reliable messaging would have been LOT of effort! Light weight, open source, met our key requirements, local support, SAP adapter was being developed, joint go-to-market strategy Claimed Ebay processes 1 billion transactions a day.. So why not?
  • Asanka
  • çèContent-Based RouterXSLTSales as per receipts IDOC: WPUBON01, aggregated: WPUUMS01Monitoring the Status of Inbound IDocs Using ALE Audit
  • Project Kick off – 1st June 2010ERP Blueprint – June 7th – July 22ndRealisation – June 28th - Oct 8thWMSGoLive – Oct 15thGolive date – 29th Feb 2011
  • WSO2Con2011: Using WSO2 ESB with SAP ERP (Retail)

    1. 1. Using WSO2 ESB with SAP ERP(Retail)byHarsha SenanayakeHead of Enterprise SolutionsJohn Keells Holdings PLC
    2. 2. Agenda• John Keells Group / SAP ERP• JKG : Enterprise Applications & Technology landscape / Vision• KeellsSuper Business requirements• Solutions Evaluated• SAP Integration technologies• Solutions Evaluated - Recommendation• Solution Architecture Implemented• Message flow and Enterprise Integration Patterns used• Project team & plan• Deployment architecture• Lessons Learnt• Q&A
    3. 3. John Keells Group• Founded in 1870; Sri Lanka‟s largest capitalized (1.6 billion USD ) company with shares listed internationally, with Global Depository Receipts (GDRs) listed on the Luxembourg Stock Exchange with a AAA+ credit rating• Diversified into key sectors of the economy - Hospitality, Transportation and Logistics, Consumer Foods, Retail, Financial services, IT, Real estate and Property development and Plantation services• 10,000+ employees• Ranked among the “200 Best under a Billion” in Asia Pacific by Forbes• Ranked the #1 Corporate in Sri Lanka by the Lanka Monthly Digest (LMD) 8 times in the recent past• Signatory to the United Nations Global Compact
    4. 4. SAP ERP• SAP (NYSE: SAP) is the market leader in enterprise application software with over 170,000 customers• The companys best known products are its enterprise resource planning application (SAP ERP), SAP BusinessObjects software, and most recently and Sybase mobile products• John Keells Holdings implemented SAP across its 70 subsidiaries in 2005 – the largest SAP implementation in Sri Lanka
    5. 5. JKG: Enterprise Applications & Technology landscapeSAP GUI WCM ESS BI dashboards Composite apps Collab Search Customer/Partners Rich / Web clients EDI /Extranet Presentation layer :Enterprise Portal Portal BEx / ODBO (OLE DB for OLAP) Flex / IGS / XMLA (MDX) EDI / Web-services / WSRP Native RFC (Java ESS apps) calls MDX Porlets (ITS, BSP, Webdynrpo) SOA governance Portlets CPM ESB DW MDM Standard ABAP - RFC data sources contents SAP Query Sync/Async Web Services XML/ IDOCS Industry specific LOB Group ERP solutions Infrastructure Business logic layer : ABAP, J2EE , .NET Directory Messaging Services Solution Lifecycle Management Identity ECM Management Infrastructure – Servers, Storage, Network, Data Center services & SSO
    6. 6. Integration vision [Gartner 2004 ] “An Enterprise Service Bus (ESB) is a new architecture that exploits  Web services,  Message-oriented-middleware (MOM)  Intelligent routing and transformation. ESBs act as a light weight, ubiquitous integration backbone through which software services and application components flow” Peace of mind : Once data is reliably delivered to the ESB, it is considered "safe" and out of the hands of the client Enterprise service bus By David A. Chappell
    7. 7. Some of the Integration scenarios @ JKHGroup – SAP related Industry specific• Async Web services postings via SAP PI (EAI) • Hotels • AR/AP/GL DFinance_Document_POST -> XI IDOC->R/3 • PMS integration with Call accounting systems/ PBX, Door locking system, Minibar systems, In room internet • Status retrieval by Polling with message GUID systems, Voice• Sync web-service interfaces via SAP PI (EAI) • CRS integration with GDS and OTAs • Credit limit check WS -> BAPI -> R/3 • Retail• Banking Interfaces • POS integration • Custom ABAP programs • 3rd party settlement integration • Payment files – MT100, Encrypted propriety forms • Transport and logistics upload to bank sites • EDI – CUSDEC • Bank reconciliation - SWIFT MT940/ MultiCash • SAP R/3 and RedPrairie WMS integration (XML<->• Direct posting / reading of data via RPCs and IDOC) WebAS Webservices • Stock brokers • RFC/BAPIs using .Net / DCOM / Jco connectors • Direct Market access via FIX • Companies – JMSL-POS, LMS-Sales app, BPOMate• Direct upload of CSV files via FTP to SAP R/3 – AR/AP/GL Upload program ZF247 • ABAP BDC uploads
    8. 8. KeellsSuper • KeellsSuper is one of the leading supermarket chains in the country which has been in operation for the last 20 years • KeellsSuper was Instrumental in popularising Modern trade in Sri Lanka • Currently at 42 outlets and expandingFirst Retailer to Only online supermarket in Sri Lanka -implement a KeellsSuper.comleading ERP in SriLanka Introduced the first Loyalty program in Retail with Integrated CRM and BI in Sri Lanka Real-time stock taking using handheld scanners
    9. 9. Business requirement• KeellsSuper has been running SAP ERP for 5 years and decided to migrate to SAP‟s ERP Retail industry solution to benefit from the rich industry specific specific functionally available in SAP Retail ERP solution
    10. 10. Business Requirement• Streamline the SAP and POS integration - Eliminate the performance issues and failures faced with the current SAP and POS integration• Seamless integration with the online store – .com
    11. 11. Business RequirementsRequired Integration scenarios Direction Description Article masters - new and changes Condition masters - price changes Vendor masters - new and changes Customer master (credit customers) Outbound Bonus Buy Conditions / Promotions (Head office -> Download phys. inv. docs, upload phys. inv. count data POS) Credit limits Stock balances Gift vouchers master data Application acknowledgements Financial transcations Inbound Upload day-end closing POS (POS - > Head Upload sales data (compressed) office) Inventory counts Gift vouchers - issues
    12. 12. Business Requirements• Guaranteed delivery - message queuing and guaranteed delivery semantics to ensure "unavailable" applications will get their data queued and delivered at a later time • Unreliable and slow networks (specially in outstations), POS Server outages• Quality of service (QOS) requirements • Exactly Once In Order – E.g. Price changes • Exactly Once – E.g. Sales data• Prioritization of Messages • Price changes Vs Article description change• Better performance without impacting the ERP system
    13. 13. Solutions evaluated1. Adopt the existing synchronous RFC based Point-to-Point integration solution developed using SAP DCOM / .Net Connector / Web services2. Develop a Point-to-point integration solution based on SAP WS-RM enabled asynchronous Web services3. Develop an IDOC based Point-to-point integration solution using SAP .Net connector4. Implement an IDOC based ESB integration solution using SAP Netweaver PI5. Implement an IDOC based ESB integration solution using WSO2 ESB
    14. 14. SAP Integration Technologies JDBC/ODBC/ADO SAP Application Server RFC SOAP SOAP / WS-RM Proces sing SAP ERPApplication RDBMS BAPI Business logic NCo (ABAP) (.net) RPC JCo IDOC (Java) IDOC EDI IDOC XML
    15. 15. Synchronous Vs Asynchronous services• With synchronous services , clients invoke a request on a service and then suspend their processing while they wait for a response from the provider. The client cannot perform another task until a response or confirmation is received from the provider.• With asynchronous services , clients initiate a request to a service and then resume their processing without waiting for a response. The service handles the client request in the provider system and returns a response at some later point, at which time the client can retrieve the response and proceed with its processing.
    16. 16. SAP Integration TechnologiesRFC/BAPIs• An RFC is a synchronous interface method that calls and executes predefined functions• A BAPI is a a RFC-enabled function that has been developed by SAP E.g : • BAPI_PO_CREATE • BAPI_PO_GETDETAIL• Synchronous RFC/BAPI calls consume SAP ERP dialog processes and slow network connections can really hog ERP system resources
    17. 17. SAP Integration TechnologiesWeb services• The ABAP Web Application Server (WebAS 6.20+) can provide existing functions (BAPIs, RFCs) as Web services• Since SAP NetWeaver 7.0, SP 14, Web Services Reliable Messaging (WS-RM) has been introduced as part of the ABAP stack. WS-RM provides reliable delivery of asynchronous messages using the SOAP protocol• SAP was an early adopter of SOA for its business applications • : The ES Workplace is the central place to view consolidated information about all available Enterprise Services delivered by SAP.
    18. 18. SAP Integration TechnologiesIDOCs• SAP Intermediate Documents (IDOCS) are EDI like documents that are asynchronous in nature.• The actual TRFC call to submit the IDOC to SAP is performed synchronously and very quickly, but the actual business processing can happen at some later time defined in the SAP system • Schedule Report RBDAPP01 as a Background Job to process the IDocs (see SAP Note 399271).• The outbound result (E.g. sales order confirmation) can also happen at some later time – Message type ALE AUDIT• IDOCS offer additional queuing and retry capabilities• IDOCs can be triggered using posting routines and Change pointers • E.g At the time of Creating Purchase Order using ME21N
    19. 19. IDOCs available for the required integrationscenariosDirection Message Type Interface description ARTMAS Article masters - new and changes COND_A Condition masters - price changes CREMAS Vendor masters - new and changes DEBMAS Customer master (credit customers)Outbound WPDBBY Bonus Buy Conditions / Promotions(Head office -> WVINVE Download phys. inv. docs, upload phys. inv. count dataPOS) ZFI_CRED Credit limits ZMM_STOCK Stock balances ZSDGV Gift vouchers master data ALEAUD Application acknowledgements WPUFIB Financial transcationsInbound WPUTAB Upload day-end closing POS(POS - > Head WPUUMS Upload sales data (compressed)office) WVINVE Inventory counts ZSDGVRE Gift vouchers - issues
    20. 20. Receiving and Sending IDOCs using .NET and Java Java IDOC Client (Sender)1.package;3.import*;4.import*;5.import*;6.import*;8.public class IDocClientExample {10.public static void main(String[] args) {12.try38.// see configuration file BCE.jcoDestination provided in the installation directory.39.JCoDestination destination=JCoDestinationManager.getDestination("BCE");40.IDocRepository iDocRepository = JCoIDoc.getIDocRepository(destination); .NET IDOC Server (Receiver)41.String tid = destination.createTID();42.IDocFactory iDocFactory = JCoIDoc.getIDocFactory();44.// a) create new idoc45.IDocDocument doc = iDocFactory.createIDocDocument(iDocRepository, "MATMAS02");46.IDocSegment segment = doc.getRootSegment();47.segment = segment.addChild("E1MARAM");48.// and so on. See IDoc Specification .....49.JCoIDoc.send(doc, IDocFactory.IDOC_VERSION_DEFAULT, destination, tid);56.destination.confirmTID(tid);58.}59.catch(Exception e)60.{61.e.printStackTrace();62.}63.System.out.print("program end");64.}65.}
    21. 21. Why IDOCs based integration was better suitedfor our requirement• Synchronous BAPIs/RFC/WS calls are resource intensive• All required WS-RM asynchronous outbound services are not available• Tried and tested technology used by almost all SAP ERP Retail customers• Only the changed data can be transferred from SAP• „POS Interface Monitor‟ functionality available to monitor IDOCs
    22. 22. Solutions evaluated – Recommendation1. Adopt the existing RFC based Point-to-PointERP dialog Synchronous RFC/BAPI/Web service calls consume SAP integration processes and slow network connections can really hog resources solution developed using SAP DCOM / .Net Connector / Point-to-point spaghetti! Web services All required WS-RM asynchronous outbound services were not available 2. Develop a Point-to-point integration solution based on Not tried and tested by SAP Retail customers, Point-to-point spaghetti! WS-RM enabled asynchronous Web services 3. Develop anwriting code for Point-to-point integration solution Made sense but IDOC based IDOC parsing , transformations and handling reliable messaging would have been LOT of effort! And why reinvent the connector using SAP .Net wheel?4. Implement an-IDOC based POS integration content out of using Separately licensed SAP provides ESB integration solution the box Dual stack (Java+ABAP) architecture makes PI very complex and heavy - SAP Netweaver PI Java only ESB was introduced with NW 7.3 5. Implement ansource, based ESBrequirements, Local support, SAP Light weight, Open IDOC Met our key integration solution using WSO2 ESB adapter was being developed, Joint go-to-market strategy WSO2 team claimed Ebay processes 1 billion transactions a day…
    23. 23. Implemented Solution architecture Asanka Abeysinghe
    24. 24. Message flow and Enterprise Integration Patterns used Example Scenario : POS Sales Data interface to SAP (2) Transform/Map to IDOC structure -WPUUMS01 has limit on max line items per IDOC therefore will need to be split into multiple  (1) Query Sales line-items by Polling IDOCs • Update on message successfully accepted by the ESB - (Update TRANFFERED_TO_ESB =“TRUE”) Polling Consumer Message translator Spilter (XSLT)  (3) Post IDOC „WPUUMS01‟ SAP ESB POSReceiver Sender  (4) Technical ack -  (5) Update on Technical ack - UPDATE IDOC created successfully /failed record : SQL messages if reprocessed in SAP. • TRANFFERED_TO_SAP_TEC_ACK=“TRUE” Content-Based Router Store and Forward  (6) Application ack - (ALEAUD)  (7) Update on Application ack - UPDATE record: (status, message) – Multiple • ALEAUD message status and update field ALEAUD messages if reprocessed in TRANSFFERED_SAP _APPLICATION_ACK = “TRUE” SAP. and update field TRANSFFERED Content-Based Router Notations from the book Enterprise Integration Patterns - Gregor Hohpe
    25. 25. Project team Head of PMO SAP Functional SAP Technical Admin / Infra POS Team WSO2 Team Core Team team Team Team SD-POS ABAP Consultant .NET and SQL Architect Process leads Consultant with IDOC skills Developers FICO Project Consultant Manager ESB MM Consultant Consultants DS Consultants QA Team
    26. 26. Project Plan
    27. 27. Project Plan Jun-10 Jul-10 Aug-10 Sep-10 Oct-10 Nov-10 Dec-10 Jan-11 Feb-11 Mar-11Project Kick off 1st JuneERP Blueprint Document POS integration scenarios Finalise IDOCs and fields to required Realisation Configure standard IDOCs Enhance/develop IDOCsWMS Blueprint RealisationPOS DevelopmentPOS Integration Solution architecture WSO2 SAP Adapter development Sizing and Deployment architecture Integration scenarios / use cases Field mapping and specifications Training Develop Transformation scripts QA / Stress testGoLive and Post Golive phase Golive preparation GoLive 29th Feb Post Golive support
    28. 28. Deployment architecture - Sizing• 500k transactions on Direction Message Type # of IDOCs on May 2011 an average month ALEAUD 4,569 ARTMAS 75,189• 1 million on seasonal COND_A 325,517 months Outbound CREMAS 214 DEBMAS 19 (Head office ->• Avg Size of a message POS) WPDBBY 646 - 100kb – 5mb WVINVE 4,110 ZFI_CRED 301 ZMM_STOCK 11,592 ZSDGV 491 WPUFIB 5,058 Inbound WPUTAB 1,339 (POS - > Head WPUUMS 8,565 office) WVINVE 33,866 ZSDGVRE 4,140
    29. 29. Deployment architectureProduction Server Configuration DRServer 1:-CPU:- 4 Cores RAM:- 8 GB Storage :- 50 GBServer 2: CPU:- 2 Cores RAM:- 4 GB Storage :- 140 GBDevelopment Server Configuration CPU:- 4 Cores RAM:- 8 GB Storage :- 105 GB
    30. 30. Lessons learnt• Avoid this:• Always revalidate requirements to avoid over engineering – E.g. Do you really need EOIO for all integration scenarios?
    31. 31. Lessons learnt• What worked in the Lab with sophisticated load test tools can break in the real environment • When you have multiple components (SAP ERP, POS, 50+ end points over unreliable networks) it’s difficult to simulate a real production environment• Strike a balance between offshore and onsite model• Have frequent project / steering committee meeting to ensure EVERYONE is on the same page to avoid surprises
    32. 32. Q&A