1
Contents
LOAD & PERFORMANCE TESTING .................................................... ERROR! BOOKMARK NOT DEFINED.
1 ST...
4.1.4 Pause Target ...................................................................................... Error! Bookmark ...
3.5 XA TRANSACTION TIMEOUT ........................................................................ ERROR! BOOKMARK NOT DE...
5.2 PURGE ...................................................................................................................
10.4 SECUREFILES.............................................................................................................
3.3 ANALYSIS OF AWR REPORT ........................................................................ ERROR! BOOKMARK NOT DE...
13 NOT KEEPING ASPECT RATIO ................................................................... ERROR! BOOKMARK NOT DEFINE...
5
6
7
8

WHAT IS THE ERROR IN LOGS? .................................................................. ERROR! BOOKMARK NOT...
1 CAPACITY PLANNING FOR BPEL PM .......................................................... ERROR! BOOKMARK NOT DEFINED.
1....
Exhibit 33: BPEL EJBs - 2 ...................................................................................................
Exhibit 73: vmstat ............................................................................ Error! Bookmark not define...
Snippets
Snippet 1: Sample Work Manager - 1 .............................................. Error! Bookmark not defined.
Sn...
Tables
Table 1: AuditLevel...................................................................................................
15
BPEL	Engine	Tuning	
In very simplistic terms, BPEL Engine is a J2EE application. This application also acts as a
container...
Exhibit 1: FMC Login

2. Summary page of Farm will appear.

Exhibit 2: FMC Farm Summary

This page shows CPU usage for Adm...
Exhibit 3: FMC Performance Summary - 1

FMC will display partitions for the managed server in left panel and few of the st...
Exhibit 4: FMC Performance Summary - 2

4. Click on “Show Metric Palette” button. Metric Palette appears at right side of ...
One can also monitor (but in real time) various matrices using DMS (Dynamic Memory Service)
servlet (http://<<host>>:<<por...
If BPEL Engine is hosting lot of synchronous processes, increasing value of
dspInvokeThreads will improve the performance ...
2.1 Dispatcher Maximum Request Depth
BPEL Engine threading model also consists of a property which controls the number of ...
Exhibit 9: Dispatcher Maximum Request Depth - 2

2.2 InvokeMessages In Cache
The MaximumNumberOfInvokeMessagesInCache prop...
3 Transaction Timeout
3.1 syncMaxWaitTime
syncMaxWaitTime specifies the maximum time the process receiver waits for gettin...
Exhibit 11: syncMaxWaitTime - 1

2. Navigate to soa-infra

SOA Administration

BPEL Properties

Exhibit 12: syncMaxWaitTim...
3. Click on "More BPEL Configuration Properties..." hyperlink

Exhibit 13: syncMaxWaitTime - 3

4. Locate syncMaxWaitTime ...
5. Click on Apply button.

3.2 BPEL EJBs
The timeout properties for the EJBs control the particular timeout setting for th...
Exhibit 16: BPEL EJBs - 2

3. Scroll down and navigate to soa-infra

EJBs.

4. Click on individual an EJB
5. Navigate “Con...
Exhibit 17: BPEL EJBs - 3

6. Using Lock & Edit, update “Transaction Timeout”
7. Click on “Save” and “Release Configuratio...
3.3 Global transaction timeout at Weblogic Domain Level
This property specifies the maximum amount of time, in seconds; an...
Exhibit 19: Global transaction timeout at WLS Domain Level - 2

4. Using Lock & Edit, update “Transaction Timeout”
5. Chan...
location="http://localhost:9010/soainfra/services/default/Child_1.0/child_client_ep?WSDL">
<property name="oracle.webservi...
Exhibit 20: WLS Timeout parameters - 1

3. Click on a particular managed server

33
Exhibit 21: WLS Timeout parameters - 2

4.

Navigate to Protocol

General tab

34
Exhibit 22: WLS Timeout parameters - 3

At this page one can tune few of the timeout parameters.
3.6.1 Complete Message Ti...
This timeout helps guard against server deadlock through too many open connections.
Minimum value: 0
Typical value: 65
3.6...
3.

Click on a particular managed server

Exhibit 24: Tunneling Client Timeout - 2

4. Navigate to Protocol

HTTP tab

37
Exhibit 25: Tunneling Client Timeout - 3

Few more parameters to tune here.
3.6.4 Post Timeout
Timeout (in seconds) for re...
Minimum value: 5
Maximum value: 3600
Typical value: 30
3.6.6 Tunneling Client Timeout
The amount of time (in seconds) afte...
Exhibit 26: SOA Infrastructure level audit tuning - 1

40
Exhibit 27: SOA Infrastructure level audit tuning - 2

4.1.1.1 Audit Level
The auditLevel property controls the audit trai...
In production environment, AuditLevel should be set to Production or Off.
4.1.1.2 Capture Composite Instance State
This at...
Set this value to <<seconds>>
policies/Element_1/properties/Element_2/name: Set this value to batchSize
policies/Element_1...
Exhibit 29: AuditConfig - 2

3.

Update attributes under AutoConfig

Exhibit 30: AuditConfig - 3

44
Exhibit 31: AuditConfig - 4

Exhibit 32: AuditConfig - 5

4. Click on Apply button.

45
4.1.2 BPEL Engine Level
4.1.2.1 AuditLevel
The auditLevel property controls the audit trail logging level. This configurat...
Exhibit 33: Payload Validation - 1

47
Exhibit 34: Payload Validation - 2

4.1.3 AuditStorePolicy
This property specifies the strategy to persist the BPEL audit ...
Exhibit 35: AuditStorePolicy - 1

Exhibit 36: AuditStorePolicy - 2

4.1.4 AuditFlushByteThreshold
This property controls h...
AuditFlushEventThreshold need to be tuned if auditStorePolicy
syncMultipleWrite or async.

is set to

4.1.6 AsyncAuditPers...
<<farmName>> WebLogic Domain
<<managedServerName>>

<<domainName>>

<<clusterName>>

Exhibit 37: Logging Tuning - 1

3. Ri...
Exhibit 38: Logging Tuning - 2

Exhibit 39: Logging Tuning - 3

4. In right hand side panel, select the dropdown named Vie...
Exhibit 40: Logging Tuning - 4

5. Expand Root Logger Node and apply log level ERROR: 1 (SEVERE)
6. Click on Apply button....
required to execute the composite.

MDS
Oracle Metadata
Services
SOA Infrastructure

MDS
SOAINFRA

Once the composite is d...
table.

DOCUMENT_CI_REF

AUDIT_TRAIL

AUDIT_DETAILS

DLV_SUBSCRIPTION

WORK_ITEM

Stores cube instance references to data ...
MDS_NAMESAPCES

this installation.
Stores name spaces referred by this deployment

Table 7: CUBE_INSTANCE

CUBE_INSTANCE

...
TITLE
STATUS
CONVERSATION_ID
PARENT_ID
ECID

8: STATE_CLOSED_ABORTED: An instance that has been aborted
due to administrat...
CONVERSATION_ID
COMPOSITE_DN
SOURCE_NAME
SOURCE_TYPE
SOURCE_ACTION_TYPE
SOURCE_ACTION_NAME
BATCH_ID
BATCH_INDEX
BUSINESS_S...
31: Completed with faulted, recovery required, suspended, and
terminated
32: Unknown
34: Open and Faulted
36: Running with...
BLOCK_CSIZE
BLOCK_USIZE
LOG

entries up to that point are written out. This block ties together
all rows written out at on...
RES_SUBSCRIBER

4: STATE_MAX_RECOVERED:
Identifier for matching subscription once found.

EXT_STRING1
EXT_STRING2
EXT_INT1...
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
BPEL PM 11g performance tuning  - 5
Upcoming SlideShare
Loading in …5
×

BPEL PM 11g performance tuning - 5

4,127 views

Published on

This is fifth chapter of Oracle Fusion Middleware BPEL PM 11g Performance Tuning. This chapter covers BPEL Engine tuning .

Published in: Technology

BPEL PM 11g performance tuning - 5

  1. 1. 1
  2. 2. Contents LOAD & PERFORMANCE TESTING .................................................... ERROR! BOOKMARK NOT DEFINED. 1 STRESS, LOAD, SOAK, SPIKE TESTS LOAD PROFILES ........................................ ERROR! BOOKMARK NOT DEFINED. 2 LNP TESTING LIFE CYCLE.......................................................................... ERROR! BOOKMARK NOT DEFINED. 3 EXAMPLES OF LNP TESTING OBJECTIVES ...................................................... ERROR! BOOKMARK NOT DEFINED. 3.1 RESPONSE TIME......................................................................................... ERROR! BOOKMARK NOT DEFINED. 3.2 RELIABILITY............................................................................................... ERROR! BOOKMARK NOT DEFINED. 3.3 CONFIGURATION SIZING .............................................................................. ERROR! BOOKMARK NOT DEFINED. 3.4 CAPACITY PLANNING .................................................................................. ERROR! BOOKMARK NOT DEFINED. 3.5 REGRESSION ............................................................................................. ERROR! BOOKMARK NOT DEFINED. 3.6 BOTTLENECK IDENTIFICATION ...................................................................... ERROR! BOOKMARK NOT DEFINED. 3.7 SCALABILITY .............................................................................................. ERROR! BOOKMARK NOT DEFINED. BPEL PM: INTRODUCTION ............................................................... ERROR! BOOKMARK NOT DEFINED. 1 BPEL PM PROCESS TYPES ....................................................................... ERROR! BOOKMARK NOT DEFINED. 1.1 BY INTERFACE ........................................................................................... ERROR! BOOKMARK NOT DEFINED. 1.2 BY DURABILITY .......................................................................................... ERROR! BOOKMARK NOT DEFINED. 2 TERMINOLOGY ...................................................................................... ERROR! BOOKMARK NOT DEFINED. 3 DIFFERENCE BETWEEN ASYNC & SYNC BPEL .............................................. ERROR! BOOKMARK NOT DEFINED. 4 SOA 11G CONFIGURATION FILES AND LOG FILES LOCATION ............................. ERROR! BOOKMARK NOT DEFINED. 5 DEFAULT DATA SOURCES ......................................................................... ERROR! BOOKMARK NOT DEFINED. 6 PERFORMANCE OBJECTIVE ....................................................................... ERROR! BOOKMARK NOT DEFINED. 7 WHAT TO TUNE .................................................................................... ERROR! BOOKMARK NOT DEFINED. 8 SYSTEM SETUP ...................................................................................... ERROR! BOOKMARK NOT DEFINED. 9 IMPORTANT POINTS ............................................................................... ERROR! BOOKMARK NOT DEFINED. 10 REFERENCE......................................................................................... ERROR! BOOKMARK NOT DEFINED. JVM TUNING ................................................................................... ERROR! BOOKMARK NOT DEFINED. 1 PAUSE TIMES AND THROUGHPUT .............................................................. ERROR! BOOKMARK NOT DEFINED. 1.1 GARBAGE COLLECTION VS. THROUGHPUT ...................................................... ERROR! BOOKMARK NOT DEFINED. 1.2 DEFRAGMENTATION PAUSES AND THROUGHPUT ............................................ ERROR! BOOKMARK NOT DEFINED. 2 PERFORMANCE AND MEMORY FOOTPRINT .................................................. ERROR! BOOKMARK NOT DEFINED. 3 JVM CHOICE ........................................................................................ ERROR! BOOKMARK NOT DEFINED. 4 JROCKIT’S TUNING ................................................................................. ERROR! BOOKMARK NOT DEFINED. 4.1 BOARDING INTO SHIP – STEP 1 .................................................................... ERROR! BOOKMARK NOT DEFINED. 4.1.1 Heap Size ............................................................................................ Error! Bookmark not defined. 4.1.2 Garbage Collection............................................................................. Error! Bookmark not defined. 4.1.3 Nursery Size ...................................................................................... Error! Bookmark not defined. 2
  3. 3. 4.1.4 Pause Target ...................................................................................... Error! Bookmark not defined. 4.2 LEAVING THE SHORE – STEP 2 ...................................................................... ERROR! BOOKMARK NOT DEFINED. 4.2.1 Lazy Unlocking ................................................................................... Error! Bookmark not defined. 4.2.2 Call Profiling ....................................................................................... Error! Bookmark not defined. 4.2.3 Large Pages ........................................................................................ Error! Bookmark not defined. 4.3 IN THE HIGH SEAS – STEP 3 ......................................................................... ERROR! BOOKMARK NOT DEFINED. 4.3.1 Compaction ........................................................................................ Error! Bookmark not defined. 4.3.2 Thread Local Area size ....................................................................... Error! Bookmark not defined. 5 HOTSPOT’S TUNING ............................................................................... ERROR! BOOKMARK NOT DEFINED. 5.1 GARBAGE COLLECTION STRATEGIES .............................................................. ERROR! BOOKMARK NOT DEFINED. 5.1.1 Serial GC Strategy .............................................................................. Error! Bookmark not defined. 5.1.2 Parallel GC Strategy ........................................................................... Error! Bookmark not defined. 5.1.3 Parallel GC strategy ............................................................................ Error! Bookmark not defined. 6 WHERE TO SPECIFY JVM AND ITS PARAMETERS ............................................ ERROR! BOOKMARK NOT DEFINED. 7 OUTOFMEMORYERROR .......................................................................... ERROR! BOOKMARK NOT DEFINED. 7.1 JAVA.LANG.OUTOFMEMORYERROR: JAVA HEAP SPACE ................................... ERROR! BOOKMARK NOT DEFINED. 7.2 JAVA.LANG.OUTOFMEMORYERROR: PERMGEN SPACE .................................... ERROR! BOOKMARK NOT DEFINED. 7.3 JAVA.LANG.OUTOFMEMORYERROR: UNABLE TO CREATE NEW NATIVE THREAD.... ERROR! BOOKMARK NOT DEFINED. 7.4 JAVA.LANG.OUTOFMEMORYERROR: REQUESTED XXX BYTES FOR CHUNKPOOL::ALLOCATE. OUT OF SWAP SPACE ERROR! BOOKMARK NOT DEFINED. 7.5 ANALYSIS ................................................................................................. ERROR! BOOKMARK NOT DEFINED. 8 REFERENCE .......................................................................................... ERROR! BOOKMARK NOT DEFINED. WEBLOGIC SERVER TUNING ............................................................ ERROR! BOOKMARK NOT DEFINED. 1 THREAD MANAGEMENT/WORK MANAGER ................................................. ERROR! BOOKMARK NOT DEFINED. 1.1 REQUEST CLASSES ..................................................................................... ERROR! BOOKMARK NOT DEFINED. 1.2 CONSTRAINTS ........................................................................................... ERROR! BOOKMARK NOT DEFINED. 1.3 WORK MANGER TYPES............................................................................... ERROR! BOOKMARK NOT DEFINED. 1.3.1 Default Thread Pool ........................................................................... Error! Bookmark not defined. 1.3.2 SOAWorkManager ............................................................................. Error! Bookmark not defined. 2 NETWORK I/O ...................................................................................... ERROR! BOOKMARK NOT DEFINED. 2.1 MUXERS .................................................................................................. ERROR! BOOKMARK NOT DEFINED. 2.2 CHUNK PARAMETERS ................................................................................. ERROR! BOOKMARK NOT DEFINED. 2.3 CONNECTION BACKLOG BUFFERING .............................................................. ERROR! BOOKMARK NOT DEFINED. 2.4 CACHED CONNECTIONS .............................................................................. ERROR! BOOKMARK NOT DEFINED. 3 TUNING DATA SOURCE CONNECTION POOLS ................................................ ERROR! BOOKMARK NOT DEFINED. 3.1 STATEMENT CACHE .................................................................................... ERROR! BOOKMARK NOT DEFINED. 3.1.1 Usage Restrictions for Statement Cache ........................................... Error! Bookmark not defined. 3.2 CONNECTION TESTING OPTIONS FOR A DATA SOURCE ..................................... ERROR! BOOKMARK NOT DEFINED. 3.2.1 Automatic Testing .............................................................................. Error! Bookmark not defined. 3.3 ENABLING CONNECTION CREATION RETRIES................................................... ERROR! BOOKMARK NOT DEFINED. 3.4 ENABLING CONNECTION REQUESTS TO WAIT FOR A CONNECTION ..................... ERROR! BOOKMARK NOT DEFINED. 3
  4. 4. 3.5 XA TRANSACTION TIMEOUT ........................................................................ ERROR! BOOKMARK NOT DEFINED. 3.6 AUTOMATICALLY RECOVERING LEAKED CONNECTIONS ..................................... ERROR! BOOKMARK NOT DEFINED. 3.7 LIMITING STATEMENT PROCESSING TIME ...................................................... ERROR! BOOKMARK NOT DEFINED. 3.8 PINNED TO THREAD PROPERTY .................................................................... ERROR! BOOKMARK NOT DEFINED. 3.9 GRIDLINK ................................................................................................. ERROR! BOOKMARK NOT DEFINED. 3.10 SINGLE CLIENT ACCESS NAME.................................................................... ERROR! BOOKMARK NOT DEFINED. 3.11 LOGGING LAST RESOURCE ......................................................................... ERROR! BOOKMARK NOT DEFINED. 4 DATABASE MODE .................................................................................. ERROR! BOOKMARK NOT DEFINED. 5 REFERENCE .......................................................................................... ERROR! BOOKMARK NOT DEFINED. BPEL ENGINE TUNING ...................................................................................................................... 16 1 REAL TIME VIEWING OF PERFORMANCE METRICS ................................................................................... 16 1.1 FUSION MIDDDLEWARE CONTROL ............................................................................................................. 16 1.2 DYNAMIC MEMORY SERVICE .................................................................................................................... 19 2 BPEL ENGINE THREADING MODEL...................................................................................................... 20 2.1 DISPATCHER MAXIMUM REQUEST DEPTH................................................................................................... 22 2.2 INVOKEMESSAGES IN CACHE .................................................................................................................... 23 3 TRANSACTION TIMEOUT ................................................................................................................... 24 3.1 SYNCMAXWAITTIME .............................................................................................................................. 24 3.2 BPEL EJBS ............................................................................................................................................ 27 3.3 GLOBAL TRANSACTION TIMEOUT AT WEBLOGIC DOMAIN LEVEL ..................................................................... 30 3.4 WEBSERVICE READ AND CONNECTION TIMEOUT .......................................................................................... 31 3.5 RELATIONSHIP AMONG TIMEOUT PARAMETERS ........................................................................................... 32 3.6 WLS TIMEOUT PARAMETERS .................................................................................................................... 32 3.6.1 Complete Message Timeout ........................................................................................................... 35 3.6.2 Idle Connection Timeout: ............................................................................................................... 35 3.6.3 Tunneling Client Timeout................................................................................................................ 36 3.6.4 Post Timeout ................................................................................................................................... 38 3.6.5 Duration .......................................................................................................................................... 38 3.6.6 Tunneling Client Timeout................................................................................................................ 39 4 BPEL PM AUDIT AND LOGGING ......................................................................................................... 39 4.1 AUDIT TUNING ....................................................................................................................................... 39 4.1.1 SOA Infrastructure level .................................................................................................................. 39 4.1.2 BPEL Engine Level ........................................................................................................................... 46 4.1.3 AuditStorePolicy ............................................................................................................................. 48 4.1.4 AuditFlushByteThreshold................................................................................................................ 49 4.1.5 AuditFlushEventThreshold .............................................................................................................. 49 4.1.6 AsyncAuditPersisterThreads ........................................................................................................... 50 4.1.7 AsyncAuditQueueSize ..................................................................................................................... 50 4.2 LOGGING TUNING................................................................................................................................... 50 5 BPEL ENGINE DATABASE SCHEMA ...................................................................................................... 53 5.1 TWO SCHEMAS ...................................................................................................................................... 53 4
  5. 5. 5.2 PURGE .................................................................................................................................................. 66 5.3 PARTITIONING........................................................................................................................................ 66 5.3.1 Partial Partitioning .......................................................................................................................... 67 5.3.2 Complete Partitioning ..................................................................................................................... 68 6 MDS SCHEMA TUNING .................................................................................................................... 68 6.1 PURGING .............................................................................................................................................. 70 6.2 STATISTIC COLLECTION ............................................................................................................................ 71 6.3 OPTIMIZING PATH_FULLNAME TABLE ........................................................................................................ 71 7 TECHNOLOGY ADAPTERS .................................................................................................................. 72 7.1 FILE AND FTP ADAPTER ........................................................................................................................... 72 7.1.1 Threads ........................................................................................................................................... 72 7.1.2 Inbound Throttling .......................................................................................................................... 73 7.1.3 Outbound Throttling ....................................................................................................................... 74 7.1.4 Outbound Performance Best Practices........................................................................................... 75 7.1.5 Tuning number of files to be processed at a time .......................................................................... 77 7.2 DATABASE ADAPTER ............................................................................................................................... 77 7.2.1 Basic considerations ....................................................................................................................... 77 7.2.2 Existence Checking.......................................................................................................................... 80 7.2.3 Throttling ........................................................................................................................................ 80 7.2.4 Adapter Threads ............................................................................................................................. 81 7.3 JMS ADAPTER ....................................................................................................................................... 82 7.3.1 Receive Threads .............................................................................................................................. 82 7.4 AQ ADAPTER ......................................................................................................................................... 83 7.4.1 Dequeue Threads ............................................................................................................................ 83 7.4.2 Dequeue Threads and Number of database connection................................................................ 83 7.5 MQ ADAPTER ........................................................................................................................................ 83 7.5.1 Dequeue Threads ............................................................................................................................ 83 7.6 SOCKET ADAPTER ................................................................................................................................... 84 8 BPEL PM – MISCELLANEOUS PROPERTIES ............................................................................................ 84 8.1 INSTANCE KEY BLOCK SIZE........................................................................................................................ 84 8.2 AUTOMATIC RECOVERY ATTEMPTS ............................................................................................................ 84 8.3 DISABLE BPEL MONITORS AND SENSORS ................................................................................................... 85 9 BEST PRACTICES: BPEL PROCESS/COMPOSITE/COMPONENT...................................................................... 85 9.1 LARGE PAYLOADS ................................................................................................................................... 85 9.2 PERSIST ONLY THE FAULTED BPEL INSTANCES.............................................................................................. 87 9.3 DO NOT VALIDATE INCOMING XML ........................................................................................................... 87 9.4 IDEMPOTENENCY .................................................................................................................................... 88 9.5 NONBLOCKINGINVOKE............................................................................................................................. 89 10 DATABASE TUNING ....................................................................................................................... 89 10.1 INITIALIZATION PARAMETERS .................................................................................................................. 89 10.2 HUGE PAGES ....................................................................................................................................... 92 10.3 DB CONTENTION.................................................................................................................................. 93 10.3.1 Wait events ................................................................................................................................... 93 5
  6. 6. 10.4 SECUREFILES........................................................................................................................................ 96 10.4.1 Enable SecureFiles for SOAINFRA schema .................................................................................... 97 10.4.2 Enabling Encryption .................................................................................................................... 101 10.5 SQL PLAN ......................................................................................................................................... 103 11 REFERENCE................................................................................................................................ 103 EM FUSION MIDDLEWARE CONTROL AND WLS ADMIN CONSOLE..... ERROR! BOOKMARK NOT DEFINED. 1 2 3 4 5 6 7 8 9 10 11 12 STUCK THREAD MAX TIME....................................................................... ERROR! BOOKMARK NOT DEFINED. STUCK THREAD TIMER INTERVAL ............................................................... ERROR! BOOKMARK NOT DEFINED. TUNE DUMP CONFIGURATION ................................................................... ERROR! BOOKMARK NOT DEFINED. DATA DISPLAY OPTION FOR EM CONSOLE ................................................... ERROR! BOOKMARK NOT DEFINED. CACHE TIMEOUT FOR DISCOVERY ............................................................... ERROR! BOOKMARK NOT DEFINED. PERMSIZE OF ADMIN SERVER ................................................................... ERROR! BOOKMARK NOT DEFINED. DMS’S FREQUENCY................................................................................ ERROR! BOOKMARK NOT DEFINED. UN-TARGET DMS FROM SERVERS ............................................................. ERROR! BOOKMARK NOT DEFINED. BPEL RECOVERY CONSOLE OPTION ............................................................. ERROR! BOOKMARK NOT DEFINED. INDEX CREATION.................................................................................. ERROR! BOOKMARK NOT DEFINED. DB MANUAL STATS COLLECTION ............................................................. ERROR! BOOKMARK NOT DEFINED. REFERENCE......................................................................................... ERROR! BOOKMARK NOT DEFINED. DATA COLLECTION .......................................................................... ERROR! BOOKMARK NOT DEFINED. 1 BEFORE LNP TEST .................................................................................. ERROR! BOOKMARK NOT DEFINED. 1.1 INFRASTRUCTURE ...................................................................................... ERROR! BOOKMARK NOT DEFINED. 1.2 JVM-HOTSPOT-<<MANAGEDSERVER>> AND JVM-JROCKIT-<<MANAGEDSERVER>> ....... ERROR! BOOKMARK NOT DEFINED. 1.3 WLS-THREADING-<<MANAGEDSERVER>> ................................................... ERROR! BOOKMARK NOT DEFINED. 1.4 WLS-NETWORKIO-<<MANAGEDSERVER>> .................................................. ERROR! BOOKMARK NOT DEFINED. 1.5 WLS-DATASOURCE-<<MANAGEDSERVER>>................................................. ERROR! BOOKMARK NOT DEFINED. 1.6 BPEL PM-<<MANAGEDSERVER>> .............................................................. ERROR! BOOKMARK NOT DEFINED. 1.7 TECHNOLOGY ADAPTERS............................................................................. ERROR! BOOKMARK NOT DEFINED. 1.8 BPEL PM – COMPOSITE ............................................................................ ERROR! BOOKMARK NOT DEFINED. 1.9 DATABASE ................................................................................................ ERROR! BOOKMARK NOT DEFINED. 2 DURING LNP TEST ................................................................................. ERROR! BOOKMARK NOT DEFINED. 2.1 CPU AND RAM USAGE AND MEMORY AVAILABILITY ........................................ ERROR! BOOKMARK NOT DEFINED. 2.2 JVM’S HEAP USAGE, GARBAGE COLLECTION AND ANY MEMORY LEAK IF ANY ........ ERROR! BOOKMARK NOT DEFINED. 2.3 DATABASE CONNECTIONS AVAILABILITY AND USAGE ........................................ ERROR! BOOKMARK NOT DEFINED. 2.4 AVAILABILITY OF PERIPHERAL RESOURCES LIKE JMS QUEUES, AQ AND THEIR USAGE PATTERN .... ERROR! BOOKMARK NOT DEFINED. 3 AFTER LNP TEST .................................................................................... ERROR! BOOKMARK NOT DEFINED. 3.1 GET FULL DUMP OF FEW IMPORTANT TABLES.................................................. ERROR! BOOKMARK NOT DEFINED. 3.2 EXECUTE SQL QUERIES AS AND WHEN REQUIRED ............................................ ERROR! BOOKMARK NOT DEFINED. 6
  7. 7. 3.3 ANALYSIS OF AWR REPORT ........................................................................ ERROR! BOOKMARK NOT DEFINED. 3.4 ANALYSIS OF SOA SERVER MEMORY USAGE ................................................... ERROR! BOOKMARK NOT DEFINED. 4 REFERENCE .......................................................................................... ERROR! BOOKMARK NOT DEFINED. LNP TEST CASE DESIGN AND LOAD TEST SELECTION ......................... ERROR! BOOKMARK NOT DEFINED. 1 LNP TEST CASE DESIGN ........................................................................... ERROR! BOOKMARK NOT DEFINED. 2 DESIGN OF EXPERIMENT .......................................................................... ERROR! BOOKMARK NOT DEFINED. 3 REFERENCE .......................................................................................... ERROR! BOOKMARK NOT DEFINED. APPENDIX A: WEBLOGIC SERVER OVERVIEW ................................... ERROR! BOOKMARK NOT DEFINED. 1 2 3 4 5 6 7 8 9 WLS CONFIGURATION ............................................................................ ERROR! BOOKMARK NOT DEFINED. DOMAIN ............................................................................................. ERROR! BOOKMARK NOT DEFINED. SERVER ............................................................................................... ERROR! BOOKMARK NOT DEFINED. ADMINISTRATION SERVER ....................................................................... ERROR! BOOKMARK NOT DEFINED. MANAGED SERVER ................................................................................ ERROR! BOOKMARK NOT DEFINED. ADMINISTRATION SERVER TO MANAGED SERVER INTERACTION ........................ ERROR! BOOKMARK NOT DEFINED. CLUSTER .............................................................................................. ERROR! BOOKMARK NOT DEFINED. NODE MANAGER................................................................................... ERROR! BOOKMARK NOT DEFINED. MACHINE ............................................................................................ ERROR! BOOKMARK NOT DEFINED. APPENDIX B: AUDITING IN BPEL PM ................................................ ERROR! BOOKMARK NOT DEFINED. 1 AUDIT LEVELS ....................................................................................... ERROR! BOOKMARK NOT DEFINED. 2 ORDER OF PRECEDENCE FOR AUDIT LEVEL SETTINGS ........................................ ERROR! BOOKMARK NOT DEFINED. APPENDIX C: ANTI PATTERNS .......................................................... ERROR! BOOKMARK NOT DEFINED. 1 2 3 4 5 6 7 8 9 10 SYNCHRONOUS – ASYNCHRONOUS ............................................................ ERROR! BOOKMARK NOT DEFINED. OVER USE OF ASYNCHRONOUS PROCESSES ................................................... ERROR! BOOKMARK NOT DEFINED. OVER USE OF DURABLE PROCESSES ............................................................. ERROR! BOOKMARK NOT DEFINED. NO FAULT HANDLING ............................................................................. ERROR! BOOKMARK NOT DEFINED. SYNCHRONOUS FAULT HANDLING .............................................................. ERROR! BOOKMARK NOT DEFINED. TO MANY RETRIES .................................................................................. ERROR! BOOKMARK NOT DEFINED. CHATTING BPEL PROCESS (CALL BACK) ....................................................... ERROR! BOOKMARK NOT DEFINED. OVER USE OF FLOWN ............................................................................. ERROR! BOOKMARK NOT DEFINED. LOOPS AND MORE LOOPS......................................................................... ERROR! BOOKMARK NOT DEFINED. SYNCHRONOUS AND ASYNCHRONOUS PROCESSES ON SAME MANAGED SERVER/CLUSTER ... ERROR! BOOKMARK NOT DEFINED. 11 DURABLE AND TRANSIENT PROCESSES ON SAME MANAGED SERVER/CLUSTER...... ERROR! BOOKMARK NOT DEFINED. 12 STICKY LOAD BALANCER ......................................................................... ERROR! BOOKMARK NOT DEFINED. 7
  8. 8. 13 NOT KEEPING ASPECT RATIO ................................................................... ERROR! BOOKMARK NOT DEFINED. APPENDIX D: SQL QUERIES .............................................................. ERROR! BOOKMARK NOT DEFINED. 1 EM CONSOLE SQL QUERIES ..................................................................... ERROR! BOOKMARK NOT DEFINED. 1.1 RECOVERY CONSOLE QUERIES ...................................................................... ERROR! BOOKMARK NOT DEFINED. 1.2 RECENT FAULT AND REJECTED MESSAGES QUERY ............................................. ERROR! BOOKMARK NOT DEFINED. 1.3 RECENT COMPOSITE INSTANCE QUERY........................................................... ERROR! BOOKMARK NOT DEFINED. 1.4 INSTANCE TAB PAGE QUERY ......................................................................... ERROR! BOOKMARK NOT DEFINED. 1.5 INSTANCE TAB PAGE SEARCH QUERY BASED ON NAME VS TITLE QUERY ................ ERROR! BOOKMARK NOT DEFINED. 1.6 FAULT AND REJECTED MESSAGE TAB PAGE QUERIES ......................................... ERROR! BOOKMARK NOT DEFINED. 1.6.1 Parent query ...................................................................................... Error! Bookmark not defined. 1.6.2 Child query ......................................................................................... Error! Bookmark not defined. 2 MISCELLANEOUS ................................................................................... ERROR! BOOKMARK NOT DEFINED. 2.1 STORED PROCEDURE TO CONVERT BLOB IN STRING .......................................... ERROR! BOOKMARK NOT DEFINED. 2.2 QUERY TO FIND PERCENTAGE OF FREE SPACE .................................................. ERROR! BOOKMARK NOT DEFINED. 2.3 QUERY TO FIND THE WAIT EVENTS FOR LGWR USING ITS SID ........................... ERROR! BOOKMARK NOT DEFINED. 2.4 QUERY TO MONITOR ‘REDO BUFFER ALLOCATION RETRIES’................................ ERROR! BOOKMARK NOT DEFINED. 2.5 SQL STATEMENT TO RECLAIM SPACE AFTER PURGING....................................... ERROR! BOOKMARK NOT DEFINED. 2.6 QUERY TO FIND OUT TOTAL SESSIONS ON A DATABASE ..................................... ERROR! BOOKMARK NOT DEFINED. 2.7 QUERY TO FIND OUT UTILIZATION OF PROCESSES AND SESSIONS IN A DATABASE ... ERROR! BOOKMARK NOT DEFINED. 2.8 FIND OUT THE PROCESS INSTANCE FROM A CONVERSATION ID WHEN THERE IS NO INSTANCE NUMBER SHOWING IN THE LOG FILE (BPEL INSTANCE ID FOR A TIMES OUT ITEM) ....................................... ERROR! BOOKMARK NOT DEFINED. 2.9 QUERY TO GET AUDIT DETAILS FROM AUDIT_DETAILS TABLE .............................. ERROR! BOOKMARK NOT DEFINED. 2.10 QUERY TO GET AUDIT DETAILS FROM AUDIT_TRAIL TABLE ............................... ERROR! BOOKMARK NOT DEFINED. 2.11 QUERY TO GET XML MESSAGE WITH THE GIVEN INSTANCE ID ......................... ERROR! BOOKMARK NOT DEFINED. 2.12 QUERY TO GET XML MESSAGE WITH A GIVEN INSTANCE NAME........................ ERROR! BOOKMARK NOT DEFINED. 2.13 QUERY TO GET PAYLOAD SIZE OF MESSAGE ................................................... ERROR! BOOKMARK NOT DEFINED. 2.14 QUERY TO GET EXECUTION TIME OF BPEL INSTANCES .................................... ERROR! BOOKMARK NOT DEFINED. 2.15 QUERY TO GET THE EXECUTION TIME OF BPEL INSTANCES AND TO FIND THE PARENT THAT HAS INITIALIZED THE COMPOSITE ....................................................................................................... ERROR! BOOKMARK NOT DEFINED. 2.16 QUERY TO IDENTIFY ALL THE FAULTS FOR THE MESSAGES THAT WERE SITTING IN BPEL ENGINE LEVEL RECOVERY AS UNDELIVERED INVOKES ........................................................................................ ERROR! BOOKMARK NOT DEFINED. APPENDIX E: BIG OR LARGE OR HUGE PAGES ................................... ERROR! BOOKMARK NOT DEFINED. 1 2 3 4 LINUX ................................................................................................. ERROR! BOOKMARK NOT DEFINED. WINDOWS ........................................................................................... ERROR! BOOKMARK NOT DEFINED. SOLARIS .............................................................................................. ERROR! BOOKMARK NOT DEFINED. REFERENCE: ......................................................................................... ERROR! BOOKMARK NOT DEFINED. APPENDIX F: ORA-01438: VALUE LARGER THAN SPECIFIED PRECISION ALLOWED .... ERROR! BOOKMARK NOT DEFINED. 8
  9. 9. 5 6 7 8 WHAT IS THE ERROR IN LOGS? .................................................................. ERROR! BOOKMARK NOT DEFINED. EFFECTS............................................................................................... ERROR! BOOKMARK NOT DEFINED. CAUSE ................................................................................................ ERROR! BOOKMARK NOT DEFINED. SOLUTION ............................................................................................ ERROR! BOOKMARK NOT DEFINED. APPENDIX G: LOBS IN THE SOAINFRA SCHEMA ................................ ERROR! BOOKMARK NOT DEFINED. APPENDIX H: AWR, ADDM, & ASH REPORTS .................................... ERROR! BOOKMARK NOT DEFINED. 1 AWR REPORT ...................................................................................... ERROR! BOOKMARK NOT DEFINED. 2 ADDM REPORT .................................................................................... ERROR! BOOKMARK NOT DEFINED. 3 ASH REPORT ........................................................................................ ERROR! BOOKMARK NOT DEFINED. 4 AWR REPORT ANALYSIS ......................................................................... ERROR! BOOKMARK NOT DEFINED. 4.1 SQL STATEMENTS ORDERED BY ELAPSED TIME ............................................... ERROR! BOOKMARK NOT DEFINED. 4.2 SQL STATEMENTS ORDERED BY CPU TIME..................................................... ERROR! BOOKMARK NOT DEFINED. 4.3 SQL STATEMENTS ORDERED BY GETS ............................................................ ERROR! BOOKMARK NOT DEFINED. 4.4 SQL STATEMENTS ORDERED BY READS .......................................................... ERROR! BOOKMARK NOT DEFINED. 4.5 SQL STATEMENTS ORDERED BY EXECUTIONS .................................................. ERROR! BOOKMARK NOT DEFINED. 4.6 SQL STATEMENTS ORDERED BY PARSE CALLS ................................................. ERROR! BOOKMARK NOT DEFINED. 5 REFERENCE .......................................................................................... ERROR! BOOKMARK NOT DEFINED. APPENDIX I: MONITORING SCRIPTS ................................................. ERROR! BOOKMARK NOT DEFINED. 1 DATABASE MONITORING ......................................................................... ERROR! BOOKMARK NOT DEFINED. 2 JMS MONITORING ................................................................................ ERROR! BOOKMARK NOT DEFINED. 3 AQ MONITORING .................................................................................. ERROR! BOOKMARK NOT DEFINED. APPENDIX J: HOW TO MONITOR SOA SERVER MEMORY USAGE ....... ERROR! BOOKMARK NOT DEFINED. 1 2 3 4 SETUP: JCONSOLE OR VISUALVM (INSTALLED LOCALLY)................................... ERROR! BOOKMARK NOT DEFINED. SETUP: JVISUALVM (INSTALLED AT REMOTE MACHINE) ................................... ERROR! BOOKMARK NOT DEFINED. SETUP: JROCKIT MISSION CONTROL (INSTALLED AT REMOTE MACHINE) .............. ERROR! BOOKMARK NOT DEFINED. REFERENCE .......................................................................................... ERROR! BOOKMARK NOT DEFINED. APPENDIX K: HEAP DUMP FILES ANALYSIS: JROCKIT AND HOTSPOT JVMS ....... ERROR! BOOKMARK NOT DEFINED. 1 EXAMPLE ANALYSIS OF A HEAP DUMP FILE USING ECLIPSE MEMORY ANALYZER ... ERROR! BOOKMARK NOT DEFINED. 2 REFERENCE .......................................................................................... ERROR! BOOKMARK NOT DEFINED. APPENDIX L: CAPACITY PLANNING................................................... ERROR! BOOKMARK NOT DEFINED. 9
  10. 10. 1 CAPACITY PLANNING FOR BPEL PM .......................................................... ERROR! BOOKMARK NOT DEFINED. 1.1 DETERMINING PERFORMANCE GOALS AND OBJECTIVES – CURRENT & FUTURE.... ERROR! BOOKMARK NOT DEFINED. 1.2 MEASURING PERFORMANCE METRICS .......................................................... ERROR! BOOKMARK NOT DEFINED. 1.3 IDENTIFYING BOTTLENECKS ......................................................................... ERROR! BOOKMARK NOT DEFINED. 1.4 IMPLEMENTING A CAPACITY MANAGEMENT PLAN .......................................... ERROR! BOOKMARK NOT DEFINED. 2 REFERENCE .......................................................................................... ERROR! BOOKMARK NOT DEFINED. Exhibits Exhibit 1: Stress, Load, Soak, Spike Tests Load Profiles ................... Error! Bookmark not defined. Exhibit 2: LnP Testing Life Cycle and SDLC ....................................... Error! Bookmark not defined. Exhibit 3: BPEL Processes ................................................................. Error! Bookmark not defined. Exhibit 4: JVM Tuning Tradeoffs ...................................................... Error! Bookmark not defined. Exhibit 5: JVM's Garbage Collection and Throughput ..................... Error! Bookmark not defined. Exhibit 6: jRockit Attributes ............................................................. Error! Bookmark not defined. Exhibit 7: HostSpot Attributes ......................................................... Error! Bookmark not defined. Exhibit 8: WLS internal applications at startup - 1 ......................... Error! Bookmark not defined. Exhibit 9: WLS internal applications at startup - 2 ......................... Error! Bookmark not defined. Exhibit 10: WLS internal applications at startup - 3 ....................... Error! Bookmark not defined. Exhibit 11: WLS internal applications at startup - 4 ........................ Error! Bookmark not defined. Exhibit 12: WLS internal applications at startup - 5 ........................ Error! Bookmark not defined. Exhibit 13: WLS internal applications at startup - 6 ........................ Error! Bookmark not defined. Exhibit 14: SOAWorkManager - 1 .................................................... Error! Bookmark not defined. Exhibit 15: SOAWorkManager - 2 .................................................... Error! Bookmark not defined. Exhibit 16: SOAWorkManager - 3 .................................................... Error! Bookmark not defined. Exhibit 17: WLS Muxers ................................................................... Error! Bookmark not defined. Exhibit 18: FMC Login.................................................................................................................... 17 Exhibit 19: FMC Farm Summary.................................................................................................... 17 Exhibit 20: FMC Performance Summary - 1 .................................................................................. 18 Exhibit 21: FMC Performance Summary - 2 .................................................................................. 19 Exhibit 22: FMC Performance Summary - 3 .................................................................................. 19 Exhibit 23: Dynamic Memory Service ........................................................................................... 20 Exhibit 24: BPEL Engine Threading Model .................................................................................... 21 Exhibit 25: Dispatcher Maximum Request Depth - 1.................................................................... 22 Exhibit 26: Dispatcher Maximum Request Depth - 2.................................................................... 23 Exhibit 27: InvokeMessages In Cache ........................................................................................... 23 Exhibit 28: syncMaxWaitTime - 1 ................................................................................................. 25 Exhibit 29: syncMaxWaitTime - 2 ................................................................................................. 25 Exhibit 30: syncMaxWaitTime - 3 ................................................................................................. 26 Exhibit 31: syncMaxWaitTime – 4 ................................................................................................. 26 Exhibit 32: BPEL EJBs - 1 ................................................................................................................ 27 10
  11. 11. Exhibit 33: BPEL EJBs - 2 ................................................................................................................ 28 Exhibit 34: BPEL EJBs - 3 ................................................................................................................ 29 Exhibit 35: Global transaction timeout at WLS Domain Level - 1 ................................................. 30 Exhibit 36: Global transaction timeout at WLS Domain Level - 2 ................................................. 31 Exhibit 37: WLS Timeout parameters - 1 ...................................................................................... 33 Exhibit 38: WLS Timeout parameters - 2 ...................................................................................... 34 Exhibit 39: WLS Timeout parameters - 3 ...................................................................................... 35 Exhibit 40: Tunneling Client Timeout - 1....................................................................................... 36 Exhibit 41: Tunneling Client Timeout - 2....................................................................................... 37 Exhibit 42: Tunneling Client Timeout - 3....................................................................................... 38 Exhibit 43: SOA Infrastructure level audit tuning - 1 .................................................................... 40 Exhibit 44: SOA Infrastructure level audit tuning - 2 .................................................................... 41 Exhibit 45: AuditConfig - 1 ............................................................................................................ 43 Exhibit 46: AuditConfig - 2 ............................................................................................................ 44 Exhibit 47: AuditConfig - 3 ............................................................................................................ 44 Exhibit 48: AuditConfig - 4 ............................................................................................................ 45 Exhibit 49: AuditConfig - 5 ............................................................................................................ 45 Exhibit 50: Payload Validation - 1 ................................................................................................. 47 Exhibit 51: Payload Validation - 2 ................................................................................................. 48 Exhibit 52: AuditStorePolicy - 1 .................................................................................................... 49 Exhibit 53: AuditStorePolicy - 2 .................................................................................................... 49 Exhibit 54: Logging Tuning - 1 ....................................................................................................... 51 Exhibit 55: Logging Tuning - 2 ....................................................................................................... 52 Exhibit 56: Logging Tuning - 3 ....................................................................................................... 52 Exhibit 57: Logging Tuning - 4 ....................................................................................................... 53 Exhibit 58: MDS Purging - 1 .......................................................................................................... 70 Exhibit 59: MDS Purging - 2 .......................................................................................................... 71 Exhibit 60: Stuck Thread Timer Interval - 1...................................... Error! Bookmark not defined. Exhibit 61: : Stuck Thread Timer Interval - 2.................................... Error! Bookmark not defined. Exhibit 62: : Stuck Thread Timer Interval - 3.................................... Error! Bookmark not defined. Exhibit 63: Data Display Option for EM Console - 1 ........................ Error! Bookmark not defined. Exhibit 64: : Data Display Option for EM Console - 2 ...................... Error! Bookmark not defined. Exhibit 65: : Data Display Option for EM Console - 3 ...................... Error! Bookmark not defined. Exhibit 66: Un-Target DMS from servers - 1 .................................... Error! Bookmark not defined. Exhibit 67: Un-Target DMS from servers - 2 .................................... Error! Bookmark not defined. Exhibit 68: : Un-Target DMS from servers - 3 .................................. Error! Bookmark not defined. Exhibit 69: Un-Target DMS from servers - 4 .................................... Error! Bookmark not defined. Exhibit 70: Un-Target DMS from servers - 5 .................................... Error! Bookmark not defined. Exhibit 71: BPEL recovery console option - 1................................... Error! Bookmark not defined. Exhibit 72: BPEL recovery console option - 2................................... Error! Bookmark not defined. 11
  12. 12. Exhibit 73: vmstat ............................................................................ Error! Bookmark not defined. Exhibit 74: free ................................................................................. Error! Bookmark not defined. Exhibit 75: pmap .............................................................................. Error! Bookmark not defined. Exhibit 76: top .................................................................................. Error! Bookmark not defined. Exhibit 77: sar-B ............................................................................... Error! Bookmark not defined. Exhibit 78: meminfo ......................................................................... Error! Bookmark not defined. Exhibit 79: mpstat ............................................................................ Error! Bookmark not defined. Exhibit 80: mpstat 2 4 ...................................................................... Error! Bookmark not defined. Exhibit 81: df-m ................................................................................ Error! Bookmark not defined. Exhibit 82: WebLogic Homes ........................................................... Error! Bookmark not defined. Exhibit 83: Synchronous – Asynchronous - 1 ................................... Error! Bookmark not defined. Exhibit 84: Synchronous – Asynchronous - 2 ................................... Error! Bookmark not defined. Exhibit 85: Database Monitoring ..................................................... Error! Bookmark not defined. Exhibit 86: JMS Monitoring .............................................................. Error! Bookmark not defined. Exhibit 87: AQ Monitoring ............................................................... Error! Bookmark not defined. Exhibit 88: Setup jConsole or visualVM (installed locally) - 1 .......... Error! Bookmark not defined. Exhibit 89: Setup jConsole or visualVM (installed locally) - 2 .......... Error! Bookmark not defined. Exhibit 90: Setup jConsole or visualVM (installed locally) - 3 .......... Error! Bookmark not defined. Exhibit 91: Setup jConsole or visualVM (installed locally) - 4 .......... Error! Bookmark not defined. Exhibit 92: Heap Dump Files analysis - 1 ......................................... Error! Bookmark not defined. Exhibit 93: Heap Dump Files analysis - 2 ......................................... Error! Bookmark not defined. Exhibit 94: Heap Dump Files analysis - 3 ......................................... Error! Bookmark not defined. Exhibit 95: Heap Dump Files analysis - 4 ......................................... Error! Bookmark not defined. Exhibit 96: Heap Dump Files analysis - 5 ......................................... Error! Bookmark not defined. Exhibit 97: Heap Dump Files analysis - 6 ......................................... Error! Bookmark not defined. 12
  13. 13. Snippets Snippet 1: Sample Work Manager - 1 .............................................. Error! Bookmark not defined. Snippet 2: Sample Work Manager - 2 .............................................. Error! Bookmark not defined. Snippet 3: Sample config.xml........................................................... Error! Bookmark not defined. Snippet 4: Sample startWeblogic.sh ................................................ Error! Bookmark not defined. Snippet 5: Using setNull In a Prepared Statement .......................... Error! Bookmark not defined. Snippet 6: Sample thread waiting for DB Call to finish.................... Error! Bookmark not defined. Snippet 7: composite.xml ............................................................................................................. 32 Snippet 8: Drop Histrogram .......................................................................................................... 71 Snippet 9: Set Table Preferences .................................................................................................. 71 Snippet 10: Single Threaded Model - jca file ................................................................................ 73 Snippet 11: Partitioned Threaded Model - jca file........................................................................ 73 Snippet 12: Tuning number of files to be processed at a time .................................................... 77 Snippet 13: JMS Adapter - Receive Thread - composite.xml ........................................................ 83 Snippet 14: AQ Adapter - Dequeue Threads - composite.xml...................................................... 83 Snippet 15: common properties schema - 1................................................................................. 86 Snippet 16: common properties schema - 2................................................................................. 86 Snippet 17: bpel.xml ..................................................................................................................... 87 Snippet 18: composite.xml ........................................................................................................... 87 Snippet 19: SecureFiles ................................................................................................................. 98 Snippet 20: BasicFiles to SecureFiles ............................................................................................ 99 13
  14. 14. Tables Table 1: AuditLevel........................................................................................................................ 41 Table 2: BPEL Engine - AuditLevel ................................................................................................. 46 Table 3: AuditStorePolicy .............................................................................................................. 48 Table 4: SOA Schemas ................................................................................................................... 53 Table 5: SOAINFRA Tables ............................................................................................................. 54 Table 6: MDS Schema tables......................................................................................................... 55 Table 7: CUBE_INSTANCE.............................................................................................................. 56 Table 8: CUBE_SCOPE ................................................................................................................... 57 Table 9: COMPOSITE_INSTANCE ................................................................................................... 57 Table 10: COMPOSITE_INSTANCE_FAULT .................................................................................... 59 Table 11: AUDIT_TRAIL ................................................................................................................. 59 Table 12: AUDIT_DETAILS ............................................................................................................. 60 Table 13: DLV_MESSAGE .............................................................................................................. 60 Table 14: DLV_SUBSCRIPTION ...................................................................................................... 61 Table 15: DOCUMENT_CI_REF ...................................................................................................... 62 Table 16: DOCUMENT_DLV_MSG_REF ......................................................................................... 62 Table 17: WORK_ITEM .................................................................................................................. 62 Table 18: XML_DOCUMENT .......................................................................................................... 65 Table 19: XML_DOCUMENT_REF .................................................................................................. 65 Table 20: HEADERS_PROPERTIES .................................................................................................. 65 Table 21: CLUSTER_MASTER ......................................................................................................... 65 Table 22: CLUSTER_NODE ............................................................................................................. 66 Table 23: Partial Partitioning ........................................................................................................ 67 Table 24: Complete Partitioning ................................................................................................... 68 Table 25: MDS_NAMESAPCES....................................................................................................... 69 Table 26: MDS_PARTITIONS ......................................................................................................... 69 Table 27: MDS_PATHS .................................................................................................................. 69 Table 28: File & FTP Adapter - Inbound Throttling ....................................................................... 73 Table 29: : File & FTP Adapter - Outbound Throttling .................................................................. 74 Table 30: : File & FTP Adapter - Outbound Best Practices............................................................ 75 Table 31: completionPersistPolicy property ................................................................................. 87 Table 32: validateSchema value ................................................................................................... 88 Table 33: Idempotency value........................................................................................................ 88 Table 34: nonBlockingInvoke value ............................................................................................. 89 Table 35: Database initialization parameters ............................................................................... 89 Table 36: Contention tables .......................................................................................................... 95 Table 37: Tables under potential of index contention ................................................................. 96 14
  15. 15. 15
  16. 16. BPEL Engine Tuning In very simplistic terms, BPEL Engine is a J2EE application. This application also acts as a container for hosting BPEL processes/composites. This application also has UI component to facilitate human interaction. While talking about tuning of BPEL we will keep this picture in view. Our focus will be tuning of BPEL engine though we will consider few points which also cover UI part as well. From tuning perspective BPEL engine can be broken into: 1. 2. 3. 4. EJBs Threading Model Database part Fusion Middleware Control ( UI part) BPEL Process Manager provides several property settings that can be configured from Enterprise Manager Fusion Middleware Control to optimize performance for response times, throughput, and concurrency. The BPEL PM can be tuned at either at the infrastructure, service engine, composite or the process level. To tune a BPEL engine, it is absolutely essential to know type of processes/composites deployed, cluster topology, WLS topology, load pattern, and JVM configuration. Processes can be categorized by the nature of their interfaces, for example, asynchronous, push based, or synchronous. They can also be durable (long running) or transient (short lived). Again, in any of these processes, there may be activities that are idempotent (retry-able, such as the Assign and Invoke activities) or invocations that are non-blocking (happening in parallel). A process can also have a breakpoint wherein its state is saved in the dehydration store or the state is not stored at all. All these factors have to be taken into consideration when tuning the BPEL PM. 1 Real Time Viewing of Performance Metrics To monitor the performance metrics at real time, one has following options. 1.1 Fusion Midddleware Control Fusion Midddleware Control (FMC) provides facility to view various matrices related to resources consumption and performance. 1. Login to FMC (http://<<serverName>>:<port>>/em) 16
  17. 17. Exhibit 1: FMC Login 2. Summary page of Farm will appear. Exhibit 2: FMC Farm Summary This page shows CPU usage for Admin server and managed server under which BPEL Engine is running. 3. Expand “SOA” from left hand side panel and invoke contextual menu by right clicking on one of the managed server. Select Monitoring Performance Summary 17
  18. 18. Exhibit 3: FMC Performance Summary - 1 FMC will display partitions for the managed server in left panel and few of the statistics in right panel. 18
  19. 19. Exhibit 4: FMC Performance Summary - 2 4. Click on “Show Metric Palette” button. Metric Palette appears at right side of screen. Exhibit 5: FMC Performance Summary - 3 From BPEL Engine and deployed BPEL Processes/composite perspective, experiment with various matrices. Once matrices of interest are identified from infrastructure set up and composites deployed perspective, observe them and note down the important observations. But FMC shows the matrices in real time only which restricts its utility from historical analysis perspective. Some time it is difficult to operate FMC during heavy load. 1.2 Dynamic Memory Service The Dynamic Monitoring Service (DMS) enables Fusion Middleware components to provide administration tools, such as Oracle Enterprise Manager, with data regarding the component's performance, state and on-going behavior. Fusion Middleware Components push data to DMS and in turn DMS publishes that data through a range of different components. Specifically, DMS is used by Oracle WebCache, Oracle HTTP Server (OHS), Oracle Application Development Framework (ADF), WebLogic Diagnostic Framework (WLDF), and JDBC. DMS measures and reports metrics trace events and system performance and provides a context correlation service for these components. 19
  20. 20. One can also monitor (but in real time) various matrices using DMS (Dynamic Memory Service) servlet (http://<<host>>:<<port>>/dms/Spy). soainfra_component is good place to start in DMS Servlet to play with. Exhibit 6: Dynamic Memory Service For historical data collection perspective one has two options: 1. Use Enterprise Manager Grid Control 2. Use custom scripts and sql queries This book will be going by second approach because it cannot be assumed that everyone has Enterprise Manger Grid in infrastructure mix. 2 BPEL Engine Threading Model BPEL Engine’s threading model is based on common sense. Before tuning it, one needs to understand the same. BPEL Engine has three thread pools. Each thread pool holds threads of specific type to manage various types of work in BPEL engine. First pool consists of system threads which does all plumbing work required to BPEL Engine keep on running. Essentially these threads do general clean up tasks like releasing stateful beans to pool. These threads are processed by BPEL Engine on high priority. Couples of these types of threads are sufficient even for heavily loaded BPEL Engine. These threads are called Dispatcher System Threads and their number is managed by dspSystemThreads property in FMC. The default value of dspSystemThreads is 2. Its value cannot be set 0 or negative. Any value less than 1 is set to default. The second pool holds Dispatcher Invoke Threads. These threads are responsible for taking any incoming message and initiating a new instance. Numbers of Dispatcher Invoke Threads in the pool are controlled by dspInvokeThreads property which can be accessed via FMC. Dispatcher Invoke Threads are also responsible for processing of synchronous processes. 20
  21. 21. If BPEL Engine is hosting lot of synchronous processes, increasing value of dspInvokeThreads will improve the performance but higher value of dspInvokeThreads also lead to greater CPU utilization due to context switching. The default value of dspInvokeThreads is 20. Its value cannot be set 0 or negative. Any value less than 1 is set to default. The third pool holds Dispatcher Engine Threads. These threads are responsible for processing asynchronous messages. Numbers of Dispatcher Engine Threads in the pool are controlled by dspEngineThreads property which can be accesses via FMC. The default value of dspEngineThreads is 30. Its value cannot be set 0 or negative. Any value less than 1 is set to default. The pictorial representation of threading model of BPEL Engine is in following exhibit: Exhibit 7: BPEL Engine Threading Model 21
  22. 22. 2.1 Dispatcher Maximum Request Depth BPEL Engine threading model also consists of a property which controls the number of inmemory activities in processes should be processed within a transaction. This property is labeled as dspMaxRequestDepth. Once the process activity chain has reached this depth, instance is dehyderated and next activity is performed in separate transaction. The default value of dspMaxRequestDepth is 600. Its value cannot be set 0 or negative. Any value less than 1 is set to default. If dspMaxRequestDepth value is set very low, number of data base calls increases which affects performance and also instance processing time may exceed application server transaction time out limit. If value is set two high, server may face out of memory error. Exhibit 8: Dispatcher Maximum Request Depth - 1 22
  23. 23. Exhibit 9: Dispatcher Maximum Request Depth - 2 2.2 InvokeMessages In Cache The MaximumNumberOfInvokeMessagesInCache property specifies the number of invoke messages that can be kept in the in-memory cache. Once the engine hits this limit, the message is pushed to dispatcher in-memory cache. The saved messages can be recovered using a recovery job. Use value -1 to disable. The default value is 100000. High value of MaximumNumberOfInvokeMessagesInCache increases performance but may result is out-of-memory issues. Exhibit 10: InvokeMessages In Cache 23
  24. 24. 3 Transaction Timeout 3.1 syncMaxWaitTime syncMaxWaitTime specifies the maximum time the process receiver waits for getting the response. If the BPEL process service component does not receive a reply within the specified time, then the activity fails. The syncMaxWaitTime property applies to durable synchronous (having breakpoint activities which forces dehyderation) processes called in a synchronous manner. A breakpoint activity is an activity in which BPEL Engine must wait for a timer to expire or for any incoming messages. The BPEL process instance is dehydrated by the BPEL Server when it encounters a breakpoint activity and the state of the process is saved in the database. The following activities are considered to be breakpoint activities: a. b. c. d. e. Receive (except if it is the first activity of the process) Wait Pick (onAlarm, onMessage) Checkpoint Reply Durable processes are those processes which have one or more breakpoint activities. Durable processes are normally long-lived. They must wait for some event to occur or some time to get expired. This parameter is only for the Synchronous process with breakpoint activities. This configuration is not for the asynchronous process since in case of async BPEL no receiver thread is waiting to timeout. Do not co-relate this parameter for the synchronous transaction to the external system from Asynchronous BPEL/composite process. Users expect a Sync invocation to external system Webservice should timeout if it’s taking more time than configure for syncMaxWaitTime period. This parameter is not applicable for this scenario, try to implement httpReadTimeout and httpConnTimeout property within composite itself If one observes following exception, consider configuring syncMaxWaitTime property: The transaction was rolled back or Transaction Rolledback.: weblogic.transaction.internal.TimedOutException: Transaction timed out 1. Login into FMC 24
  25. 25. Exhibit 11: syncMaxWaitTime - 1 2. Navigate to soa-infra SOA Administration BPEL Properties Exhibit 12: syncMaxWaitTime - 2 25
  26. 26. 3. Click on "More BPEL Configuration Properties..." hyperlink Exhibit 13: syncMaxWaitTime - 3 4. Locate syncMaxWaitTime and change it. Exhibit 14: syncMaxWaitTime – 4 26
  27. 27. 5. Click on Apply button. 3.2 BPEL EJBs The timeout properties for the EJBs control the particular timeout setting for the SOA application, overriding the global setting specified by the JTA timeout (See step 3). Note: Prior to changing BPEL EJBs transaction time out values, ensure to shutdown SOA managed server. Otherwise errors will popup. 1. Log into WLS Administration Console. Exhibit 15: BPEL EJBs - 1 2. Click “Deployments” 27
  28. 28. Exhibit 16: BPEL EJBs - 2 3. Scroll down and navigate to soa-infra EJBs. 4. Click on individual an EJB 5. Navigate “Configuration” Tab 28
  29. 29. Exhibit 17: BPEL EJBs - 3 6. Using Lock & Edit, update “Transaction Timeout” 7. Click on “Save” and “Release Configuration”. 8. Repeat above steps for following EJBs: BPELActivityManagerBean BPELDeliveryBean BPELDispatcherBean BPELEngineBean BPELFinderBean BPELInstanceManagerBean BPELProcessManagerBean BPELSensorValuesBean BPELServerManagerBean 9. Start SOA Managed Server 29
  30. 30. 3.3 Global transaction timeout at Weblogic Domain Level This property specifies the maximum amount of time, in seconds; an active transaction is allowed to be in the first phase of a two-phase commit transaction. If the specified amount of time expires, the transaction is automatically rolled back. In case of single phase transactions, if the transaction is still in the "active" state after this time, it is automatically rolled back. Minimum value: 1 Maximum value: 2147483647 1. Log into WLS Administration Console. Exhibit 18: Global transaction timeout at WLS Domain Level - 1 2. Click on “JTA” 3. Navigate to Configuration JTA tab 30
  31. 31. Exhibit 19: Global transaction timeout at WLS Domain Level - 2 4. Using Lock & Edit, update “Transaction Timeout” 5. Change the value of Timeout Seconds. 6. Click on “Save” and “Release Configuration”. 7. Restart WLS. 3.4 Webservice Read and connection timeout When an http request has been made to external system using web service httpReadTimeout, httpConnTimeout, and optimization properties can be used to timeout that transaction. These values are identified in milliseconds and this configuration only applicable for http protocol based web service invocation. One can define httpReadTimeout, httpConnTimeout, and optimization in composite.xml for the service for which timeout has to set. <reference name="Service1" ui:wsdlLocation="http://localhost:90101/soainfra/services/default/Child_1.0/Child.wsdl"> <interface.wsdl interface="http://xmlns.oracle.com/test/Child/Child#wsdl.interface(Child)"/> <binding.ws port="http://xmlns.oracle.com/test/Child/Child#wsdl.endpoint(child_client_ep/ Child_pt)" 31
  32. 32. location="http://localhost:9010/soainfra/services/default/Child_1.0/child_client_ep?WSDL"> <property name="oracle.webservices.local.optimization">false</property> <property name="oracle.webservices.httpReadTimeout" type="xs:string" many="false">10000</property> <property name="oracle.webservices.httpConnTimeout" type="xs:string" many="false">5000</property> </binding.ws> </reference> Snippet 1: composite.xml oracle.webservices.local.optimization parameter value could be true/false, make it false if service which user is going to invoke reside in same WLS Instance to avoid execution of SOAP stack In the BPEL process one can add a Catch branch for a RemoteFault if wants to catch this error. Note: When using this parameter it will return a "bpelx:timeout" Fault, and not a RemoteFault. 3.5 Relationship among Timeout parameters If the invocation pattern is like this: Request - Consumer invokes SOA Load balancer SOA Managed Server Provider System Response - Consumer invokes  OA Load balancer  SOA Managed Server  Provider System Then as a general rule, keep the following relation between the timeout parameters: Consumer system/API Timeout > SOA Load Balancer Timeout (If applicable) > Global Transaction Timeout JTA > BPEL EJB's transaction timeout > syncMaxWaitTime OR composite Level Timeout > Provider System/API timeout Note: This recommendation is only applicable to sync Processes. Additionally the default Timeout setting that comes with SOA 11g installation does not comply with this rule. One might need to adjust the setting according to particular business and technical requirements. 3.6 WLS Timeout parameters There are few more parameters related to timeout which may also affect performance. Let us consider first at general protocol level for a managed server. 1. Login into WLS console 2. Navigate to <<Domain Name>> Environment Servers 32
  33. 33. Exhibit 20: WLS Timeout parameters - 1 3. Click on a particular managed server 33
  34. 34. Exhibit 21: WLS Timeout parameters - 2 4. Navigate to Protocol General tab 34
  35. 35. Exhibit 22: WLS Timeout parameters - 3 At this page one can tune few of the timeout parameters. 3.6.1 Complete Message Timeout CompleteMessageTimeout defines the maximum number of seconds that this server waits for a complete message to be received. Each network channel can override this value if configured. CompleteMessageTimeout affects the HTTP Response, such that if WLS discovers sockets inactive for longer than the CompleteMessageTimeout, the server will close these sockets. Though original intent of this attribute is guard against a denial of service attack but can be used to timeout long messages to improve throughput of WLS. Minimum value: 0 Maximum value: 480 3.6.2 Idle Connection Timeout: The maximum number of seconds that a connection is allowed to be idle before it is closed by the server. IdleConnectionTimeout defines the maximum number of seconds that a connection is allowed to be idle before it is closed by the server. The T3 and T3S protocols ignore this attribute. Each network channel can override this value if configured 35
  36. 36. This timeout helps guard against server deadlock through too many open connections. Minimum value: 0 Typical value: 65 3.6.3 Tunneling Client Timeout The amount of time (in seconds) after which a missing tunneled client is considered dead. Each network channel can override this value if configured Minimum value: 1 Let us move to HTTP tab: 1. Login into WLS console 2. Navigate to <<Domain Name>> Environment Servers Exhibit 23: Tunneling Client Timeout - 1 36
  37. 37. 3. Click on a particular managed server Exhibit 24: Tunneling Client Timeout - 2 4. Navigate to Protocol HTTP tab 37
  38. 38. Exhibit 25: Tunneling Client Timeout - 3 Few more parameters to tune here. 3.6.4 Post Timeout Timeout (in seconds) for reading HTTP POST data in a servlet request. If the POST data is chunked, the amount of time the server waits between the end of receiving the last chunk of data and the end of receiving the next chunk of data in an HTTP POST before it times out. Though original intent of this attribute is guard against a denial of service attack but can be used to timeout long messages to improve throughput of WLS. Minimum value: 0 Maximum value: 120 Typical value: 30 3.6.5 Duration The amount of time this server waits before closing an inactive HTTP connection. Number of seconds to maintain HTTP keep-alive before timing out the request. 38
  39. 39. Minimum value: 5 Maximum value: 3600 Typical value: 30 3.6.6 Tunneling Client Timeout The amount of time (in seconds) after which a missing tunneled client is considered dead. Each network channel can override this value if configured. Minimum value: 1 4 BPEL PM Audit and Logging In BPEL Engine, Audit and Logging are clearly demarcated. Auditing answers the question: • Who did what and why? Logging is focuses on: • What's happening and how? Auditing is about recording domain-level events: a transaction is created; messages under processing, etc. Audit trail is recorded in database. Audit trail is important from business perspective. Logging means the recording of implementation level events that happen as the BPEL Engine is running: such as general health of BPEL Engine, any exception/error thrown by BPEL Engine or process/composite, etc. Logging is recorded in file system. Generally technical team is more interested in logging. 4.1 Audit Tuning Audit information is used for viewing the state of the process from FMC. 4.1.1 SOA Infrastructure level The properties set at SOA Infrastructure level impacts all deployed composites, except those composites for which explicitly different audit or payload validation values are set at the composite application level. a. Log in to FMC b. Navigate to <<farmName>> SOA <<managedServer>> SOA Administration Common Properties 39
  40. 40. Exhibit 26: SOA Infrastructure level audit tuning - 1 40
  41. 41. Exhibit 27: SOA Infrastructure level audit tuning - 2 4.1.1.1 Audit Level The auditLevel property controls the audit trail logging level. This configuration property is applicable to both durable and transient processes. AuditLevel property has following possible values: Table 1: AuditLevel Value Production Development Off Description Composite instance tracking is collected, but Mediator engine will not collect payload details and BPEL engine will not collect payload details for assign activities (payload details for other BPEL activities are collected). This level is optimal for most normal production operations. This is default value. Allows both the composite instance tracking and payload detail tracking. However it may impact the performance. This level is useful mostly for testing and debugging purposes. No logging is performed. Composite instance tracking and payload details are not collected. 41
  42. 42. In production environment, AuditLevel should be set to Production or Off. 4.1.1.2 Capture Composite Instance State This attributes enables tracking for "running" instances. Enabling this feature may impact the performance negatively. 4.1.1.3 Payload Validation Selecting this field validates incoming and outgoing XML documents using Schema validation. Nonschema-compliant payload data is intercepted and displayed as a fault. This property is applicable to both durable and transient processes. Enabling XML payload validation can impact performance negatively. 4.1.1.4 AuditConfig This attribute is collection of few properties but only following are important from performance perspective. a. instanceTrackingAuditTrailThreshold: This property sets the maximum size (in kilobytes) of an audit trail details string which will be stored in audit_trail table. If audit trail string is larger than instanceTrackingAuditTrailThreshold then it stored in audit_details table. Data from audit_details is not immediately loaded when the audit trail is initially retrieved in FMC but a hyper link is displayed which on click shows the string stored. instanceTrackingAuditTrailThreshold value affect performance of BPEL Engine while processing messages but also UI component (FMC) as well. b. When to audit: By default, audit data is logged to the database synchronously. By switching the audit policy to deferred, all audit operations are invoked asynchronously Capture Composite Instance State must be disabled. Set following values: policies/Element_0/name: Set this value to Immediate policies/Element_0/isActive: policies/Element_1/name: Set this value to false Set this value to Deferred policies/Element_1/isActive: set this value to True Set this value to maxThreads policies/Element_1/properties/Element_0/value: Set this value to <<number>> policies/Element_1/properties/Element_1/name: Set this value to flushDelay policies/Element_1/properties/Element_0/name: 42
  43. 43. Set this value to <<seconds>> policies/Element_1/properties/Element_2/name: Set this value to batchSize policies/Element_1/properties/Element_2/value: Set this value to <<number>> policies/Element_1/properties/Element_1/value: 1. Click on More SOA Infra Advanced Configuration Properties... Exhibit 28: AuditConfig - 1 2. Click on AutoConfig 43
  44. 44. Exhibit 29: AuditConfig - 2 3. Update attributes under AutoConfig Exhibit 30: AuditConfig - 3 44
  45. 45. Exhibit 31: AuditConfig - 4 Exhibit 32: AuditConfig - 5 4. Click on Apply button. 45
  46. 46. 4.1.2 BPEL Engine Level 4.1.2.1 AuditLevel The auditLevel property controls the audit trail logging level. This configuration property is applicable to both durable and transient processes. AuditLevel property has following possible values: Table 2: BPEL Engine - AuditLevel Value Inherit Off Minimal Error Production Development Description Inherits the audit level from infrastructure level. No audit events (activity execution information) are persisted and no logging is performed All events are logged; however, no audit details (variable content) are logged. Logs only serious problems that require immediate attention from the administrator and are not caused by a bug in the product. All events are logged. The audit details for assign activities are not logged; the details for all other activities are logged. All events are logged; all audit details for all activities are logged. Performance improvement is observed if auditLevel is set to off or Minimal or Error. 4.1.2.2 AuditDetailThreshold The auditdetailthreshold property sets the maximum size (in kilobytes) of an audit trail details string which will be stored in audit_trail table. If audit trail string is larger than auditdetailthreshold then it stored in audit_details table. Data from audit_details is not immediately loaded when the audit trail is initially retrieved in FMC but a hyper link is displayed which on click shows the string stored. . The default value is 50000 (50 kilobytes). auditdetailthreshold value affect performance of BPEL Engine while processing messages but also UI component (FMC) as well. 4.1.2.3 Payload Validation validateXML setting to true or selecting Payload Validation checkbox validates incoming and outgoing XML documents using Schema validation. Nonschema-compliant payload data is intercepted and displayed as a fault. This property is applicable to both durable and transient processes. Enabling XML payload validation can impact performance negatively. 46
  47. 47. Exhibit 33: Payload Validation - 1 47
  48. 48. Exhibit 34: Payload Validation - 2 4.1.3 AuditStorePolicy This property specifies the strategy to persist the BPEL audit data. Table 3: AuditStorePolicy Value Description Audit Trail and dehydration are persisted to DB in one transaction. syncMultipleWrite Audit Trail and dehydration are persisted in the same thread but separate transactions. async Audit Trail and dehydration are persisted by separate threads and separate transactions. syncSingleWrite By default, audit messages are stored as part of the main BPEL transaction. In scenarios where a large number of audit logs are accumulated as part of a single transaction like looping, may lead to an out-of-memory and/or time out issue for BPEL main transaction. One may consider using syncMultipleWrite or async to store the audit message separately from the main transaction. When using syncMultipleWrite or async, for auditStorePolicy one needs tune AuditFlushByteThreshold and AuditFlushEventThreshold properties as well. 48
  49. 49. Exhibit 35: AuditStorePolicy - 1 Exhibit 36: AuditStorePolicy - 2 4.1.4 AuditFlushByteThreshold This property controls how often the engine should flush the audit events, after adding an event to the current batch, the engine checks to see if the current batch byte size is greater than this value or not. AuditFlushByteThreshold need to be tuned if auditStorePolicy syncMultipleWrite or async. is set to 4.1.5 AuditFlushEventThreshold This property controls how often the engine should flush the audit events, when it reaches to mentioned limit of the number of events, the engine would trigger a call to persist audit data. 49
  50. 50. AuditFlushEventThreshold need to be tuned if auditStorePolicy syncMultipleWrite or async. is set to 4.1.6 AsyncAuditPersisterThreads When auditStorePolicy is set to async, the audit data will be put in an in-memory queue and a separate pool of that performs the persistence of the audit data. This property sets size of this pool. AsyncAuditPersisterThreads value needs to be tuned based on how much audit log being generated and CPU utilization. 4.1.7 AsyncAuditQueueSize When auditStorePolicy is set to async, the audit data will be put in an in-memory queue. AsyncAuditQueueSize sets upper bound of the queue, controlling maximum number of entries in the queue. AsyncAuditQueueSize value needs to be tuned based on how much audit log being generated and free memory. 4.2 Logging Tuning There are two types of loggers: 1. Persistence Logger 2. Runtime Logger Persistent loggers become active when the component is started. The log levels for these loggers are persisted across component restarts. Runtime loggers are automatically created during runtime and become active when a particular feature area is exercised. BPEL Engine records logs in flat files on the file system. The logs typically reside under $MW_HOME/<<project>> /domains/<<domainName>>/servers/<<ServerName>>/logs. Set the log levels to ERROR: 1 (SEVERE) across the board. This will reduce the amount of information logged and will give a slight improvement on performance. These log levels can be adjusted and enabled to higher levels of auditing at runtime, if additional debugging information is ever required. To set the log levels: 1. Log in to FMC. 2. Navigate till 50
  51. 51. <<farmName>> WebLogic Domain <<managedServerName>> <<domainName>> <<clusterName>> Exhibit 37: Logging Tuning - 1 3. Right click on one of the managed server and navigate till Log Configuration <<managedServerName>> Logs Log Configuration 51
  52. 52. Exhibit 38: Logging Tuning - 2 Exhibit 39: Logging Tuning - 3 4. In right hand side panel, select the dropdown named View with value With Persistent Log Level State. 52
  53. 53. Exhibit 40: Logging Tuning - 4 5. Expand Root Logger Node and apply log level ERROR: 1 (SEVERE) 6. Click on Apply button. 5 BPEL Engine Database Schema The SOA Suite database requires sufficient memory to be available in order to support the enterprise class installation. To approximate the memory requirements on the machine where the database is installed, the following demands should be taken into consideration: • • • • • Oracle Database overhead. Size of System Global Area Number of concurrent users Any non-database software that has to run on the machine. Growth in usage over planned lifetime of SOA Suite BPEL Engine and deployed composites/processes are transaction intensive applications, tuning of underlying database results in major performance boost. 5.1 Two Schemas SOA Suite uses two database schemas. Table 4: SOA Schemas Component BPEL Process Manager Schema SOAINFRA Description Contains the design and run-time repository of BPEL PM and BPM. The design repository has modeling metadata and profile data for composites. This data describe the behavior of composites and sequence of steps 53
  54. 54. required to execute the composite. MDS Oracle Metadata Services SOA Infrastructure MDS SOAINFRA Once the composite is deployed, the run-time repository contains the metadata to execute the composite. Contains process definitions and configuration reference data. Contains metadata for composites that use MDS. Contains metadata related to B2B, BPEL PM, Workflow, Sensor, Mediator, and CEP. Tables in SOAINFRA Schema do not enforce referential integrity; Components of SOA Suite manages referential integrity via code. Within SOAINFRA schema following are important tables used by SOAP Infrastructure and BPEL PM: Table 5: SOAINFRA Tables Fusion Component Table COMPOSITE_INSTANCE COMPOSITE_INSTANCE_FAULT Description Stores information related to composites. Stores stack trace for system level exceptions for composites. SOA Infrastructure column error_category represents business and technical faults. REFERENCE_INSTANCE COMPOSITE_SENSOR_VALUE COMPONENT_INSTANCE REJECTED_MESSAGE REJECTED_MSG_NATIVE_PAYLOA D INSTANCE_PAYLOAD COMPOSITE_INSTANCE_ASSOC CLUSTER_NODE BPEL Process Manager CLUSTER_MASTER CUBE_INSTANCE Stores cluster related information on which BPEL PM is running. Stores cluster’s master node info. Stores the processes instance metadata for every instance that is being initiated, under execution and executed successfully/faulted. CI_INDEXES CUBE_SCOPE Stores the scope data of an instance. All the variables declared in the BPEL flow and some internal objects to help route logic throughout the flow are stored in this 54
  55. 55. table. DOCUMENT_CI_REF AUDIT_TRAIL AUDIT_DETAILS DLV_SUBSCRIPTION WORK_ITEM Stores cube instance references to data stored in the XML_DOCUMENT table. Stores the audit trail for instances which can be viewed in FMC. As an instance is executed, each activity writes events to the audit trail as XML. Stores audit details that can be logged through the API. Activities such as an assign activity log the variables as audit details by default. If instanceTrackingAuditTrailThreshold is set audit trail is stored here. Stores delivery subscriptions for an instance. Whenever an instance expects a message from a partner (for example, receive or onMessage activity) a subscription is written out for that specific receive activity. Stores each activity’s metadata in an instance. This table includes the metadata for the activity (current state, label, and expiration date (used by wait activities)). AUDIT_COUNTER WI_FAULT DLV_MESSAGE HEADERS_PROPERTIES DOCUMENT_DLV_MSG_REF XML_DOCUMENT Stores incoming (invocation) and callback messages metadata upon receipt. It also stores pending messages. Stores headers and properties information. Stores references to DLV_MESSAGE documents stored in the XML_DOCUMENT table. Stores all large objects in the system (e.g. DLV_MESSAGE documents). This table stores the data as binary large objects (BLOBs). Separating the document storage from the metadata enables the metadata to change frequently without being impacted by the size of the documents. Table 6: MDS Schema tables Fusion Component Table MDS MDS_PARTITIONS MDS_PATHS Description Stores details of SOA components installed using this repository Stores details of components/composites deployed on 55
  56. 56. MDS_NAMESAPCES this installation. Stores name spaces referred by this deployment Table 7: CUBE_INSTANCE CUBE_INSTANCE Column CIKEY CREATION_DATE MODIFY_DATE STATE Description Primary Key. Acts as Foreign Key (logically but not physically) in lot of other tables Date Time stamp indicating initiated date time of an instance. Date Time stamp to indicate latest change in an instance Current state of an instance. 0: STATE_INITIATED: An instance that has just been created. 1: STATE_OPEN_RUNNING: An instance that has been created and has active activities executing. The instance is not in an exception or error condition. 2: STATE_OPEN_SUSPENDED: An instance that is unavailable. Performers of any of the activities that belong to this instance cannot take any action until the instance has returned to the running state. 3: STATE_OPEN_FAULTED: An instance that has an activity that has thrown an exception. When an activity throws an exception, the instance is flagged as being in an exception state until the exception is bubbled up, caught and handled. 4: STATE_CLOSED_PENDING_CANCEL: An instance that has started its cancellation procedure. Since cancelling an instance may involve a great deal of business logic, the amount of time the entire cancellation process may take from seconds to days. During this time, the instance is said to be pending cancellation; an instance may not be acted upon during this time. 5: STATE_CLOSED_COMPLETED: An instance that has been completed. All activities belonging to this instance have also been completed. 6: STATE_CLOSED_FAULTED: An instance that has an activity that has thrown an exception while the instance is being cancelled. This state is equivalent to STATE_OPEN_FAULTED except that when the exception is resolved, the state transitions back to CLOSED_PENDING_CANCEL rather than STATE_OPEN_RUNNING 7: STATE_CLOSED_CANCELLED: An instance that has been cancelled. All activities belonging to this instance have also been cancelled. 56
  57. 57. TITLE STATUS CONVERSATION_ID PARENT_ID ECID 8: STATE_CLOSED_ABORTED: An instance that has been aborted due to administrative control. All activities belonging to this instance are also moved to the aborted state. 9: STATE_CLOSED_STALE: An instance who's process has been changed since the process was last accessed. No actions may be performed on the instance. All activities that belong to this instance are also moved to the stale state 10: STATE_CLOSED_ROLLED_BACK Current instance title (as specified in composite, no engine impact) Current status (as specified in composite) Identifier associated with instance, e.g. passed in via WSAddressing or user specified custom key CMPST_ID or CIKEY of the parent instance that created this instance, instance at the top of the tree will null value Foreign key (logical only not physical)from COMPOSITE_INSTANCE table COMPOSITE_INSTANCE.ECID CMPST_ID COMPONENTTYPE COMPOSITE_NAME DOMAIN_NAME COMPONENT_NAME COMPOSITE_REVISION CREATE_CLUSTER_NODE_ID Name of composite Application name Name of component Version number of composite Node ID on which this composite if executed. CLUSTER_NODE.IP_ADDRESS CPST_INST_CREATED_TIME Table 8: CUBE_SCOPE CUBE_SCOPE Column CIKEY Description It is primary key. This column ensures one to one foreign key relationship from CUBE_INSTANCE.CIKEY BINARY_FORMAT MODIFY_DATE SCOPE_BIN Date scope last modified Scope bytes Table 9: COMPOSITE_INSTANCE COMPOSITE_INSTANCE Column ECID Description Primary Key ID PARENT_ID 57
  58. 58. CONVERSATION_ID COMPOSITE_DN SOURCE_NAME SOURCE_TYPE SOURCE_ACTION_TYPE SOURCE_ACTION_NAME BATCH_ID BATCH_INDEX BUSINESS_STATUS TITLE TAGS STATE 0: Running 1: Completed 2: Running with faults 3: Completed with faults 4: Running with recovery required 5: Completed with recovery required 6: Running with faults and recovery required 7: Completed with faults and recovery required 8: Running with suspended 9: Completed with suspended 10: Running with faults and suspended 11: Completed with faults and suspended 12: Running with recovery required and suspended 13: Completed with recovery required and suspended 14: Running with faults, recovery required, and suspended 15: Completed with faults, recovery required, and suspended 16: Running with terminated 17: Completed with terminated 18: Running with faults and terminated 19: Completed with faults and terminated 20: Running with recovery required and terminated 21: Completed with recovery required and terminated 22: Running with faults, recovery required, and terminated 23: Completed with faults, recovery required, and terminated 24: Running with suspended and terminated 25: Completed with suspended and terminated 26: Running with faulted, suspended, and terminated 27: Completed with faulted, suspended, and terminated 28: Running with recovery required, suspended, and terminated 29: Completed with recovery required, suspended, and terminated 30: Running with faulted, recovery required, suspended, and terminated 58
  59. 59. 31: Completed with faulted, recovery required, suspended, and terminated 32: Unknown 34: Open and Faulted 36: Running with recovery required and unknown state 64: Stale LIVE_INSTANCES STATE_COUNT HAS_ASSOC VERSION Version of composite Table 10: COMPOSITE_INSTANCE_FAULT COMPOSITE_INSTANCE_FAULT Column ECID ID Description COMPOSITE_INSTANCE.ECID PARENT_ID COMPOSITE_INSTANCE_ID COMPOSITE_DN UNIQUE_ID BATCH_ID CREATED_TIME client SERVICE_NAME BINDING_TYPE binding.ws PROTOCOL_MESSAGE_ID PROTOCOL_CORRELATION_ID ERROR_CATEGORY s d ERROR_CODE ERROR_MESSAGE STACK_TRACE RETRY_COUNT MESSAGE_ORIGIN_REFERENCE ADDITIONAL_PROPERTIES Table 11: AUDIT_TRAIL AUDIT_TRAIL Column CIKEY COUNT_ID BLOCK Description CUBE_INSTANCE.CIKEY Many audit trail entries may be made for each instance; this column is incremented for each entry per instance. When the instance is dehydrated, the batched audit trail 59
  60. 60. BLOCK_CSIZE BLOCK_USIZE LOG entries up to that point are written out. This block ties together all rows written out at one time. Compressed size of block in bytes Uncompressed size of block in bytes Block bytes NUM_OF_EVENTS Table 12: AUDIT_DETAILS AUDIT_DETAILS Column CIKEY DETAIL_ID Description CUBE_INSTANCE.CIKEY BLOCK_CSIZE BLOCK_USIZE Part of composite key, means of identifying particular detail from the audit trail Compressed size of detail in bytes Uncompressed size of detail in bytes DOC_REF BIN Detail bytes Table 13: DLV_MESSAGE DLV_MESSAGE Column CONV_ID CONV_TYPE MESSAGE_GUID Description Conversation id (correlation id) for the message. This value is used to correlate the message to the subscription. internal use Unique identifier for the message...each message received by the engine is tagged with a message guid. PARTNER_LINK OPERATION_NAME Operation name for callback port. EVENT_NAME RECEIVE_DATE STATE date message was received by engine Current state of message 0: STATE_UNRESOLVED: State value for a message/subscriber inserted into system but has not been correlated with its corresponding subscriber/message. 1: STATE_RESOLVED: State value for a message/subscriber has been correlated with its corresponding subscriber/message but has not been processed by the BPEL domain (callback). 2: STATE_HANDLED: State value for a message/subscriber has been processed by the BPEL domain (callback). 3: STATE_CANCELLED: State value for a message/subscriber has been cancelled, will not be processed by the BPEL process. 60
  61. 61. RES_SUBSCRIBER 4: STATE_MAX_RECOVERED: Identifier for matching subscription once found. EXT_STRING1 EXT_STRING2 EXT_INT1 DLV_TYPE 1: Invoke Message: New Message 2: DLV Message: Callback Message MASTER_CONV_ID PRIORITY COMPOSITE_NAME DOMAIN_NAME COMPONENT_NAME COMPOSITE_LABEL COMPOSITE_REVISION COMPONENT_TYPE CIKEY RECOVER_COUNT HEADER_PROPERTIES_BIN_FORMAT ECID CLUSTER_NODE_ID CLUSTER_NODE_KEY CACHE_VERSION PROPERTIES HEADERS_REF_ID Table 14: DLV_SUBSCRIPTION DLV_SUBSCRIPTION Column CONV_ID CONV_TYPE Description Conversation id for subscription, used to help correlate received delivery messages. internal use CIKEY PARTNER_LINK CUBE_INSTANCE.CIKEY PROCESS_GUID guid for process this subscription belongs to operation name for subscription (receive, onMessage operation name) Null The work item composite key that this subscription is positioned at (key for receive, onMessage work item). internal use date subscription was created Current state of subscription 0: STATE_UNRESOLVED: State value for a message/subscriber inserted into system but has not been 61 OPERATION_NAME EVENT_NAME SUBSCRIBER_ID SERVICE_NAME SUBSCRIPTION_DATE STATE

×