Describes some of the lessons learned while implementing high-performance, near-real time integration using SAP PI. This was presented at the Mastering SAP Technologies 2011 conference in Sydney, Australia

2. Recipes For the Perfect PI - Simple Ingredients for Complex Requirements SaschaWenninger - Australia Post

3. Bottom-Up Approach Photo by Gidzy

4. You’ll Hear About: Basis Adapter Framework Interface Design Questions Key Points to Take Home

5. 30,000 Foot View SAP POS DM SAP PI SAP ERP Routing Mapping SAP SCEM XML XML XML XML Mapping Mapping JMS Queue Mapping Mapping … …

6. Some Figures… Retail Transactions from 8,000+ terminals in 3,400 stores: 500,000 – 750,000 messages per day Peaks of 45/second into PI, then split by receiver 6 receiving systems, 1-3 receivers per message Message size <10kB Roll-out: January – June 2011

7. Objectives Near-real time: <15 minutes end to end Scalable to 2,000,000 messages/day and 14 receiver systems Future peaks of 90/second into PI No impact on other critical PI interfaces e.g. Parcel Tracking interface moving 2m+ events per day As simple as possible Future-proof (no ccBPM, no ABAP mapping, etc.)

8. The Solution? Many small steps enabled us to get there.

9. Key Topic Basis

10. Basis System Memory Database Stuff Java Server Nodes The JVM Java Heap Space Wily Introscope

11. System Memory One of the main foundations of the system Needs to accommodate all aspects of the system: Java + ABAP + ICM + DBMS + OS + caching/buffers + other apps Need to avoid swapping to disk at all times – causes huge GC delays!

13. The JVM PI 7.1 and later use SAP’s own JVM SAPJVM 4.1 for PI 7.0 coming soon (Notes 1495160, 1522198) Bug fixes and performance improvements delivered continually Several JVM Upgrades over the past 12 months significantly reduced OOM crashes and Garbage Collection times for our systems. This enables the use of larger heap spaces! Patching via JSPM takes 30 minutes and 1 restart, plus some testing

14. Java Heap Space More Heap = process larger messages, fewer full GCs, fewer OOMs SAP’s recommendation has historically been 2GB per Java process. Recently changed to 4GB (for SAP JVM) AusPost has been using 4GB for 9 months now. Main criteria: Full GCs should take less than 10 seconds.

15. Java Memory Analysis  This is in SolMan!

16. Wily Introscope Monitoring for the Java Stack Basic version is free with NetWeaver Java Provides functionality normally accessed via ABAP transactions

17. Wily Dashboards

18. More Dashboards

19. And More…

20. Tons of Dashboards! …and this is only the free version!

21. Our PI System PI 7.11 SP5 JVM 5.1 Patch level 59 (64 in test systems) 3 Java server nodes, 4GB heap space each. AIX 6.1 on IBM POWER6, 60GB RAM

23. Database Stuff

24. Java Server Nodes

25. The JVM

26. Java Heap Space

27. Wily Introscope Many aspects which require special PI-Basis skills!

28. Key Topic Adapter Framework

29. Adapter Framework Thread Behaviour Messaging System

30. Threads Messages are processed using a number of adapter-specific queues. Each queue has its own thread pool. e.g. for JMS:

32. Caveat Some Adapters process messages serially! e.g. JMS: 1 Adapter Thread per Communication Channel reads messages from the JMS queue { 1 JMS Comm Channel 3 Java Server Nodes

33. Threads in the AAE Adapter Thread picks up a message from the JMS queue, then: Executes Adapter Modules Performs Receiver Determination Persists 1 message per Receiver in Messaging System queue. Confirms message and removes from the JMS queue The longer this takes, the lower throughput will be.

34. What does this mean?

35. Sender-Side Performance is Really Important

36. Implications Throughput depends on Sender Adapter performance Try to avoid adapter modules in Sender adapters. We achieved a 15% improvement by moving an XSLT mapping from the JMS adapter to the Operation Mapping.

39. 2 Options In our system, each thread can process 10-15 messages/sec In order to consistently process 45 messages/sec from JMS, we need 4-5 concurrent JMS Sender Adapter threads. Option 1: 5 Java server nodes Option 2: Multiple JMS Communication Channels.

41. More scalableThis is not needed for SOAP since it’s truly parallel. Image Source: http://t.co/ZOBdlH0

43. Messaging System Increased importance for Integrated Configuration scenarios!

44. Key Topic Interface Design

45. Integrated Configuration Receiver Determination + Interface Determination + Receiver Agreements Integrated Configuration Object Causes an interface to be executed in the Advanced Adapter Engine

46. Integrated Configuration Message Processing entirely in Java (AAE) Introduced in PI 7.1; more features in 7.11 and 7.3 Improved performance by: Reducing database I/O Eliminating ‘stack jumping’ between ABAP and Java 7-10 times throughput possible Not available for all adapters and scenarios.

47. Example

48. Example

49. Constraints There are a few things you can’t do with Integrated Configuration: IDocs* Multi-Mappings* ccBPMs But this is an Opportunity to be Creative! *features added in PI 7.3

51. Communication Channels ABAP Proxies/Enterprise Services  Use SOAP Adapter in XI 3.0 mode JMS Senders  Clone comm. Channels for greater parallelisation Non-SAP Systems  Ensure inbound queuing is available to avoid backlog in PI

52. Design-Time Governance Don’t Overdo It! Find a balance which facilitates reuse but doesn’t impose too much up-front work. Image Source: http://geekandpoke.typepad.com

53. Questions

54. Key Points to Take Home Wily Introscope Essential for understanding performance of the Java stack Use Integrated Configuration Objects 20 minutes to migrate an existing interface Keep your JVM Current Continuous improvements without much fuss Most of the other points can be quick wins too if you don’t do them all at once.

55. SaschaWenninger Australia Post sascha.wenninger@auspost.com.au @sufw +61 403 933 472

56. Further Reading PI Performance Check SOA MiddlewareArticles on SDN Incl. the Performance section