Double the Performance of Oracle SOA Suite 11g? Absolutely!

Double the Performance of Oracle SOA Suite 11g?
Absolutely!
April 10th, 2013
12:15-12:45
Mile High Ballroom 3C
Session ID #187
Ahmed Aboulnaga
Technical Director
Raastech, Inc.

of 44 © Raastech, Inc. 2013 | All rights reserved.
1. Introduction
2. Summary
3. Monitoring Performance
4. Tuning
5. Resources
Agenda

INTRODUCTION

 Ahmed Aboulnaga
 15 years of Oracle middleware experience
 Focus on Oracle Fusion Middleware, with emphasis on SOA,
portal, and identity management technologies
 Implementations for defense, commercial, federal,
international, and global customers
 Author of “Oracle SOA Suite 11g Administrator’s Handbook”
 OCE (SOAArchitect/SOAFoundations) and OCA (ApplicationServer)
 President of WMOUG/EGOUG
 Oracle ACE
 Blogger at blog.raastech.com
About Me

SUMMARY

Summary
 JVM heap size is rarely a concern if:
• You size it appropriately
• You are running on PS3 (11.1.1.4) or higher
 Disk I/O is never a concern on the midtier… the database is
a different story
 SWAP is rarely a concern, but keep your eyes on it
 CPU utilization is usually never a problem unless the
AdminServer acts up

Summary
 Upgrading to PS3 (11.1.1.4) addresses memory instability issues
 Moving from Sun JDK to JRockit results in a 32% boost
 Optimizing the audit level makes a huge difference, resulting in
a 46% to 92% performance improvement
 Increasing the Mediator worker threads results in a 30%
performance improvement for async services
 Mediator sequential routing rules may improve performance
from 4% to 509%
 Don’t forget to tune the BPEL Service Engine
 Consider parallel garbage collection, but don't have statistics

MONITORING PERFORMANCE

Instance Performance
 Timestamps on the console display up to seconds
 Timestamps on the console are “last update time”

SQL Queries
 SQL queries to the dehydration store are the best way to get
performance related metrics

 DMS Spy Servlet displays instant Dynamic Monitoring
Service (DMS) related metrics
 Navigate to http://<host>:<soaport>/dms/Spy
http://docs.oracle.com/cd/E15586_01/core.1111/e10108/monitor.htm#CFAHIAIB
The DMS Spy Servlet

Java Heap
 Navigate to Home > Monitoring Dashboard
 Once sized appropriately, usually not a problem

Data Sources
 Navigate to Home > Data Sources

root@soahost1:/root> df –m
Filesystem 1M-blocks Used Available Use% Mounted on
/dev/sda8 996 451 494 48% /
/dev/sda9 815881 697454 76314 91% /u01
/dev/sda7 996 36 909 4% /home
/dev/sda5 1984 138 1744 8% /tmp
/dev/sda3 1984 283 1598 16% /var
/dev/sda2 5950 3842 1802 69% /usr
/dev/sda1 99 12 83 13% /boot
tmpfs 8023 0 8023 0% /dev/shm
Disk Space
 Often an underlooked area

 The vmstat command easily outputs CPU, memory, and I/O
statistics
 Do not rely on Linux’s reporting of available memory, and
best to look at SWAP space usage
 Why Linux reports 100% memory usage all the time:
http://blog.raastech.com/2008/01/why-linux-reports-100-memory-usage-all.html
Check CPU, RAM, and I/O
root@soahost1:/root> vmstat -S m
procs -------memory--------- --swap-- ---io-- --system-- ----cpu-----
r b swpd free buff cache si so bi bo in cs us sy id wa st
0 0 0 59 402 15055 0 0 2 16 0 0 2 2 96 1 0

 System log files can reveal resource issues:
Check OS Resources
root@soahost1:/root> cat /var/log/messages
Aug 31 20:53:22 uslx286 sshd[22480]: fatal:
setresuid 10000: Resource temporarily unavailable
root@soahost1:/root> ps -A | wc -l
297
root@soahost1:/root> lsof | wc -l
6064
 Too many open files can exhaust system resources:
 Too many running processes can exhaust system resources:

TUNING

Tuning the Java Virtual Machine (JVM)
What?
 Switch from Sun JDK to JRockit JDK
 64-bit, to avoid 3GB (Linux) or 4GB (Windows) limitation
Why?
 32% overall performance improvement
 Forget what other people say about benefits of Sun JDK

Tuning the Java Virtual Machine (JVM)
How?
$MW_HOME/wlserver_10.3/common/bin/commEnv.sh
OLD: JAVA_HOME="/u01/app/oracle/jdk1.6.0_26"
NEW: JAVA_HOME="/u01/app/oracle/jrockit1.6.0_29"
OLD: JAVA_VENDOR=Sun
NEW: JAVA_VENDOR=Oracle
$MW_HOME/user_projects/domains/[Domain]/bin/setDomainEnv.sh
OLD: BEA_JAVA_HOME=""
NEW: BEA_AVA_HOME="/u01/app/oracle/jrockit1.6.0_29"
OLD: SUN_JAVA_HOME="/u01/app/oracle/jdk1.6.0_26"
NEW: SUN_JAVA_HOME=""
$MW_HOME/user_projects/domains/[Domain]/bin/setSOADomainEnv.sh
OLD: PORT_MEM_ARGS="-Xms768m -Xmx1536m"
NEW: PORT_MEM_ARGS="-Xms1536m -Xmx1536m -Xgcprio:throughput
-XXtlasize:min=16k,preferred=128k,wasteLimit=8k"

Tuning the Java Transaction API (JTA)
What?
 JTA provides begin/commit/rollback functionality for apps
 The default value is 30 seconds
Why?
 Want to avoid errors such as
Unexpected exception while enlisting XAConnection
java.sql.SQLException: Transaction rolled back:
Transaction timed out after 301 seconds

Tuning the Java Transaction API (JTA)
How?
 Navigate to Services > JTA
 Set Timeout Seconds from 120 to 300

Tuning the Log Level
What?
 Standard SOA Suite logs are located in:
$MW_HOME/user_projects/domains/[Domain]/
servers/[soa_server]/logs
Why?
 Excessive logging results in considerably more disk space
usage and also impacts performance

Tuning the Log Level
How?
 Right-click on soa-infra, navigate to Logs > Log Configuration
 Select the dropdown View With Persistent Log Level State
Log Levels - oracle.bpm ERROR: 1 (SEVERE)
Log Levels - oracle.integration ERROR: 1 (SEVERE)
Log Levels - oracle.sdp ERROR: 1 (SEVERE)
Log Levels - oracle.sdpinternal ERROR: 1 (SEVERE)
Log Levels - oracle.soa ERROR: 1 (SEVERE)
Log Levels - oracle.sysman ERROR: 1 (SEVERE)
Log Levels - oracle.wsm ERROR: 1 (SEVERE)

Tuning the Audit Store Logging
What?
 Disable Capture Composite Instance State and
Payload Validation should be disabled
Why?
 24% performance improvement
 Because must query the COMPOSITE_INSTANCE,
CUBE_INSTANCE, and MEDIATOR_INSTANCE tables

Tuning the Audit Store Logging
How?
 Right-click on soa-infra, navigate to SOA Administration >
Common Properties
 Uncheck Capture Composite Instance State
 Uncheck Payload Validation

Tuning the Audit Level
What?
 Reduce the audit levels to Production
Why?
 Reducing the audit level from Development to Production
results in a 46% performance improvement
 Changing the audit level from Production to Off results in a
further 61% performance improvement, but Off is not
recommended

Tuning the Audit Level
How?
Common Properties
 Set Audit Level to Production

Tuning the Audit Configuration Policy
What?
 Set the audit configuration to Deferred instead of the default
Immediate setting
Why?
 Perhaps the largest composite instance performance gain
can be attained by setting the audit policy to deferred, which
results in all audit data being committed to the database
asynchronously

Tuning the Audit Configuration Policy
How?
Common Properties > More SOA Infra Advanced
Configuration Properties
 Click on Audit Config

Tuning the BPEL Service Engine
What?
 There are many BPEL Service Engine properties that can be
tuned, but we will go through some of the key ones
Why?
 Trust me!

Tuning the BPEL Service Engine
Parameter Name Default Tuned Remarks
Audit Trail 50000 20000 Messages with values greater than this threshold are
saved in the AUDIT_DETAILS table instead of
AUDIT_TRAIL.
Dispatcher System
Threads
2 10 Cleaning and housekeeping work for the engine.
Dispatcher Engine
Threads
30 100 Responsible for picking and processing instances. Set
higher if a lot of processes with many dehydration
points.
Dispatcher Invoke
Threads
20 70 Instantiates new instances in the engine. Tune higher
value if there are multiple branches/flows for faster
processing. Basically throttles the load setting of your
BPEL Engine.
AuditStorePolicy SyncLocal
Transaction
AsyncsyncL
ocal
Transaction
Will change the audit data storage strategy to save audit
data asynchronously, using a separate local transaction.
SyncMaxWaitTime 45 300 Determines the timeout for request-response
operations.

Tuning the Mediator Service Engine
What?
 There are many Mediator Service Engine properties that can
be tuned, but we will go through some of the key ones
Why?
 To improve overall throughput
 Increasing the Mediator worker threads can result in a 30%
performance improvement for async services

Tuning the Mediator Service Engine
How?
Mediator Properties
Parameter Name Default Tuned Remarks
Metrics Level Enabled Disabled Unless DMS metrics collection is
explicitly required.
Parallel Worker Threads 4 20 Small vs. large environments.
Parallel Maximum
Rows Retrieved
200 600 Small vs. large environments.
Parallel Locker Thread Sleep 2 1 Bad parameter. Always set to 1.

Implementing BPEL In Memory Persistence
What?
 Set inMemoryOptimization in your BPEL processes
Why?
 To reduce the number of times the BPEL process writes to
the dehydration store
 Useful for certain high performance use cases

Tuning BPEL Persistence
How?
 Add the following to composite.xml to stop in-flight
instances from writing to the database:
<BPELProcess src="YourProcess.bpel" id="YourProcess">
<configurations>
<property name="inMemoryOptimization">true</property>
<property name="completionPersistPolicy">faulted</property>
<property name="completionPersistLevel">all</property>
</configurations>
</BPELProcess>

Implementing HTTP Timeouts
What?
 Set HTTP timeouts at design time
Why?
 To reduce consumption of threads in the event of poor
performing external services

Tuning HTTP Timeouts
How?
 Add the following two properties in the reference section of
composite.xml:
<reference name="Register">
<binding.ws port="http://tempuri.org/#wsdl.endpoint
location="RegisterWS.svc.wsdl">
<property name="oracle.webservices.httpReadTimeout"
type="xs:string" many="false">10000</property>
<property name="oracle.webservices.httpConnTimeout"
type="xs:string" many="false">10000</property>
</binding.ws>
</reference>
 Can be manipulated at runtime

Using Mediator Sequential Routing Rules
What?
 ‘Sequential’ Mediator routing rules are superior in
performance compared to ‘parallel’
 This is because of the poor design that Oracle has chosen
for avoiding thread starvation
Why?
 Implementing sequential routing rules may improve
performance of Mediator anywhere from 4% to 509%

100 messages98 messages
Wait 1 second
 No messages here
Wait 1 second
Wait 1 second
99 messages

Using Mediator Sequential Routing Rules
How?
 Edit all *.mplan files for your Mediator operations and
change the executionType as follows:
OLD: executionType="direct"
NEW: executionType="queued"

RESOURCES

 Oracle SOA Suite 11g Administrator’s
Handbook
http://www.packtpub.com/oracle-soa-suite-11g-
administrators-handbook/book
 Chapter 4: Tuning Oracle SOA Suite
11g for Optimum Performance
 “Highly recommended, a tour de
force.”
~Mark Nelson, Oracle A-Team
Book
http://redstack.wordpress.com/2012/10/28/a-review-of-oracle-soa-suite-11g-administrators-handbook/

 Performance tuning Oracle SOA Suite 11g
http://blog.raastech.com/2011/04/performance-tuning-oracle-soa-
suite-11g.html
Blog

Ahmed Aboulnaga
Technical Director
ahmed.aboulnaga@raastech.com
Contact Information

Double the Performance of Oracle SOA Suite 11g? Absolutely!

More Related Content

What's hot

Viewers also liked

Similar to Double the Performance of Oracle SOA Suite 11g? Absolutely!

More from Revelation Technologies

Recently uploaded

Double the Performance of Oracle SOA Suite 11g? Absolutely!