<Insert Picture Here> Performance Analysis <ul><li>Overview </li></ul><ul><li>Examples </li></ul><ul><li>Tooling </li></ul>
Performance Analysis Overview <ul><li>Goals are to understand detailed “whats”, and “whys” </li></ul><ul><ul><li>For whom ...
General Performance Analysis Approach Describe > Hypothesize > Evaluate > Recommend > Confirm <ul><li>Get a precise descri...
Typical Path for Heap Problems Seen in Traditional J2EE Apps and Integration Apps Java VM unresponsive Hypothesis: GC-boun...
Analyzing Heap Issues Look from VM Perspective, not OS Perspective 13900.575: [Full GC 1854335K->1687971K(1975744K), 10.51...
Analyzing Heap Issues First Level of Tooling: jstat -t -gc <pid> 60s Timestamp  S0C  S1C  S0U  S1U  EC  EU  OC  OU  PC  PU...
Diagnosing Memory Leaks Second Level of Tooling: What’s on the Heap and Why? <ul><li>Want heap snapshots / instance histog...
-XX:+PrintClassHistogram Output And Comparison Between Successive Outputs <ul><li>num  #instances  #bytes  class name </li...
JADE Heap Analysis
JADE Heap Comparison
Analyzing Response Time Breakdown <ul><li>Key to optimization: where is response time coming from? </li></ul><ul><li>10.1....
BPEL Console Statistics Page
ESB Control Instances View Statistics
OWSM Monitor See Chapter 6 of OWSM Administrator’s Guide
EM Transaction Tracing
Analyzing Response Time Breakdown <ul><li>Key to optimization: where is response time coming from? </li></ul><ul><li>10.1....
Example Response Time Analysis Where is the Time Coming From? <ul><li>select domain, process, count(*), round(avg(time_tak...
Example Response Time Analysis From Monitoring to Analysis <ul><li>We now knew what was slow, and how slow </li></ul><ul><...
Example Response Time Analysis Correlated with Load – Endpoint Overloaded?
Creative Capabilities Embed Timing Instrumentation in BPEL Process <ul><li><bpelx:exec name=&quot;assign_invoke_start_time...
Creative Capabilities Embed Timing Instrumentation in BPEL Process <ul><li>Yields the following output in BPEL JVM output ...
Example Response Time Analysis Endpoint Response Time Distribution
Example Response Time Analysis Root Cause Identification <ul><li>Endpoint vendor’s measurements differed from ours </li></...
<Insert Picture Here> Performance Optimization <ul><li>Overview </li></ul><ul><li>Examples </li></ul><ul><li>Principles, P...
Performance Optimization <ul><li>There is no silver bullet one-size-fits-all prescription </li></ul><ul><li>Every situatio...
BPEL Application Architecture NAÏVE! BPEL Host BPEL Server Must be  product defect!
BPEL Application Architecture Clustering & Load Balancing for Scalability BPEL Host BPEL Server BPEL Host BPEL Server BPEL...
BPEL Application Architecture Pattern: Tier per Pipeline Stage BPEL Host BPEL Server BPEL Host BPEL Server BPEL Host BPEL ...
Principles – Response Time <ul><li>Minimize inter-process communication </li></ul><ul><ul><li>Co-location not distribution...
Principles - Throughput <ul><li>Network of queues model, queuing theory </li></ul><ul><li>Queues build upstream of slow se...
Principles – Heap Utilization <ul><li>Reduce  thread count – fewer threads to concurrently instantiate objects on heap </l...
Patterns <ul><li>SOA patterns is a nascent field – choose wisely </li></ul><ul><ul><li>Start with Gregor Hohpe’s survey </...
Product Tuning Playbooks <ul><li>BPEL </li></ul><ul><ul><li>OOW 2006 BPEL performance presentation (S282687) </li></ul></u...
<Insert Picture Here> Summary, Q&A
Summary <ul><li>APM is a professional responsibility of app ops/dev teams </li></ul><ul><li>Adopt tooling for monitoring, ...
Summary Prospects for Standarization Collect, store, and chart standard measures Yes Montoring At a high level, it’s a dev...
A Q & Q U E S T I O N S A N S W E R S
For More Information search.oracle.com +&quot;application performance management&quot; +&quot;service-oriented architectur...
 
 
Upcoming SlideShare
Loading in …5
×

Soa Performance Patterns

3,334 views

Published on

Performance management for SOA applications - Randy Stafford

Performance of SOA applications is a serious concern for their owners and architects, especially as usage and complexity increase over the life of an application. The fact is that attending to quality attributes like performance, scalability, and availability will always be part of architecting systems, even as technology platforms evolve and new development models emerge over the years.

This presentation will review the fundamental aspects of a pragmatic performance management approach for operational SOA applications, borne of first-hand experience in responsible positions. And it will drill into specialties such as performance analysis including understanding resource utilization and response time breakdown, and performance optimization using offline request stream replay and patterns specific to applications of different types and architectural styles.

Attendees will gain a working knowledge of concepts, vocabulary, and measures from the world of performance management, in categories such as load, performance, and resource utilization. They will see examples of the kinds of charts and other artifacts that are useful in performance analysis activities, and learn of simple techniques and tools they can use to initiate a performance management program for their own systems.

Attendees need not meet any special prerequisites, but an appreciation of evidence-based decision-making would be helpful. Ultimately, the more we are able to describe and manage the performance of our applications using common measures, the more mature we will be as a profession.

Published in: Business, Technology
  • Be the first to comment

Soa Performance Patterns

  1. 1. <Insert Picture Here> Performance Analysis <ul><li>Overview </li></ul><ul><li>Examples </li></ul><ul><li>Tooling </li></ul>
  2. 2. Performance Analysis Overview <ul><li>Goals are to understand detailed “whats”, and “whys” </li></ul><ul><ul><li>For whom is performance poor, and when, and exactly how poor? </li></ul></ul><ul><ul><li>Where does the major response time contribution come from? </li></ul></ul><ul><ul><li>Where is the throughput bottleneck? </li></ul></ul><ul><ul><li>Why is this happening? </li></ul></ul><ul><ul><li>Only then can you know how best to optimize </li></ul></ul><ul><li>Usually initiated by a report of some trouble </li></ul><ul><ul><li>Better initiated proactively as ongoing APM activity </li></ul></ul><ul><li>Benefits from the application of the scientific method </li></ul><ul><ul><li>First, lay down some monitoring </li></ul></ul><ul><ul><li>Then, begin generating and evaluating hypotheses… </li></ul></ul><ul><li>Requires another level of tooling </li></ul>
  3. 3. General Performance Analysis Approach Describe > Hypothesize > Evaluate > Recommend > Confirm <ul><li>Get a precise description of the situation </li></ul><ul><ul><li>Exactly what is observed, and when it is/not observed </li></ul></ul><ul><ul><li>Example: “The server crashes.” The host reboots? A process terminates unexpectedly? A process becomes unresponsive? At startup? Under certain load? After a certain duration of operation? </li></ul></ul><ul><li>Formulate hypotheses about cause </li></ul><ul><ul><li>This uses your experience, engineering judgment, intuition </li></ul></ul><ul><li>Evaluate hypotheses by collecting and examining evidence </li></ul><ul><ul><li>Configuration files, log files, metrics, etc. </li></ul></ul><ul><ul><li>You may have to instrument & reproduce </li></ul></ul><ul><ul><li>You’re trying to confirm or reject hypotheses </li></ul></ul><ul><li>With confirmed hypothesis, recommend resolution </li></ul><ul><ul><li>This uses product knowledge, architectural knowledge </li></ul></ul><ul><li>After resolution implementation, confirm it worked via evidence </li></ul><ul><li>Beware of skipping steps: Describe > Hypothesize > Evaluate > Recommend > Confirm </li></ul>
  4. 4. Typical Path for Heap Problems Seen in Traditional J2EE Apps and Integration Apps Java VM unresponsive Hypothesis: GC-bound Monitor heap, GC activity Recommend tuning, redesign Note this is not the only possible cause! Param chg Too small PermGen OutOfMemError Redesign GC-bound Unresponsive Heap over-consumption CPU-bound Blocked on IFs Deadlock Possible Cause Redesign OutOfMemError Upgrade? Unresponsive Redesign Unresponsive Redesign Unresponsive Likely Fix Symptom Unexpected termination StackOverflowError Frequent major GC OutOfMemoryError Manifestation Infinite recursion Infinite recursion Heap over-consumption Heap over-consumption Problem
  5. 5. Analyzing Heap Issues Look from VM Perspective, not OS Perspective 13900.575: [Full GC 1854335K->1687971K(1975744K), 10.5195922 secs] 13911.456: [Full GC 1854335K->1696808K(1975744K), 8.1486279 secs] 13920.094: [Full GC 1854335K->1690473K(1975744K), 8.2035046 secs] 13928.622: [Full GC 1854335K->1697287K(1975744K), 8.1587776 secs] 13937.049: [Full GC 1854335K->1688773K(1975744K), 10.4990567 secs] 13948.293: [Full GC 1854335K->1699080K(1975744K), 8.2193787 secs] 13956.974: [Full GC 1854335K->1692996K(1975744K), 8.2299517 secs] 13965.561: [Full GC 1854335K->1700780K(1975744K), 8.2348413 secs] 13973.968: [Full GC
  6. 6. Analyzing Heap Issues First Level of Tooling: jstat -t -gc <pid> 60s Timestamp S0C S1C S0U S1U EC EU OC OU PC PU YGC YGCT FGC FGCT GCT 71126.4 1280.0 1408.0 584.1 0.0 171648.0 113746.6 932096.0 422943.4 109568.0 109239.4 2538 54.318 1165 1165.298 1219.616 71186.4 1408.0 1600.0 1336.1 0.0 171520.0 165054.3 932096.0 423536.4 109568.0 109245.6 2560 54.988 1166 1167.631 1222.619 71246.4 1728.0 1856.0 544.0 0.0 170816.0 117204.8 932096.0 423600.7 109568.0 109261.4 2584 55.720 1167 1169.948 1225.668 71306.4 1600.0 1600.0 0.0 1048.6 171136.0 145917.6 932096.0 418206.3 109568.0 109160.3 2599 56.182 1168 1172.607 1228.789 71366.5 1600.0 1536.0 0.0 1000.1 171456.0 63392.6 932096.0 422258.1 109568.0 109175.3 2617 56.744 1169 1174.944 1231.688 71426.4 1600.0 1536.0 0.0 1529.7 171584.0 45124.1 932096.0 422541.6 109568.0 109354.1 2619 56.811 1170 1177.296 1234.108 71486.5 1600.0 2112.0 0.0 0.0 170496.0 32226.8 932096.0 378758.2 109824.0 109359.3 2620 56.846 1171 1179.798 1236.643
  7. 7. Diagnosing Memory Leaks Second Level of Tooling: What’s on the Heap and Why? <ul><li>Want heap snapshots / instance histograms after major GC </li></ul><ul><li>-XX:+PrintClassHistogram </li></ul><ul><ul><li>On SIGQUIT (kill -3), forces GC, then prints histogram to stdout </li></ul></ul><ul><ul><li>SIGQUIT VM periodically, compare histograms to ID instance growth </li></ul></ul><ul><ul><li>Pros: </li></ul></ul><ul><ul><ul><li>Lightweight, low-impact way to gain initial insight into leak </li></ul></ul></ul><ul><ul><li>Cons: </li></ul></ul><ul><ul><ul><li>Broken with default collector in JDK 5.0 </li></ul></ul></ul><ul><ul><ul><li>Won’t tell how instances are reachable </li></ul></ul></ul><ul><li>Enterprise Manager Java Application Diagnostic Expert (JADE) </li></ul><ul><ul><li>Acquired Auptyma, a JVMTI-based Java application profiler </li></ul></ul><ul><ul><li>The ultimate recommended solution </li></ul></ul><ul><ul><li>Simple agent install; analysis uses larger install and heap snapshots </li></ul></ul>
  8. 8. -XX:+PrintClassHistogram Output And Comparison Between Successive Outputs <ul><li>num #instances #bytes class name </li></ul><ul><li>-------------------------------------- </li></ul><ul><li>1: 26753 206538760 [B </li></ul><ul><li>2: 2264478 181158240 com.collaxa.cube.xml.dom.CubeDOMElement </li></ul><ul><li>3: 2551791 115425128 [Ljava.lang.Object; </li></ul><ul><li>4: 1030051 81715400 [C </li></ul><ul><li>5: 1835933 44062392 com.collaxa.cube.xml.dom.CubeDOMAttribute </li></ul><ul><li>6: 1741524 41796576 java.util.ArrayList </li></ul><ul><li>7: 1736747 41681928 java.lang.String </li></ul><ul><li>8: 815454 39141792 org.apache.xerces.dom.ElementImpl </li></ul><ul><li>9: 727890 29115600 org.apache.xerces.dom.AttrNSImpl </li></ul><ul><li>10: 130525 20910656 <methodKlass> </li></ul><ul><li>11: 51697 19844016 [Ljava.util.HashMap$Entry; </li></ul><ul><li>12: 784107 18818568 java.util.Vector </li></ul><ul><li>13: 783155 18795720 org.apache.xerces.dom.AttributeMap </li></ul><ul><li>14: 35912 12851944 [I </li></ul><ul><li>15: 258748 8279936 org.collaxa.thirdparty.dom4j.QName </li></ul><ul><li>16: 159477 7333608 <symbolKlass> </li></ul><ul><li>17: 11481 6857008 <constantPoolKlass> </li></ul><ul><li>18: 87374 5808152 [J </li></ul><ul><li>19: 178734 5719488 oracle.xml.parser.v2.XMLAttr </li></ul><ul><li>20: 176691 5654112 oracle.xml.parser.v2.XMLElement </li></ul><ul><li>21: 11481 5157504 <instanceKlassKlass> </li></ul><ul><li>22: 10037 4062384 <constantPoolCacheKlass> </li></ul><ul><li>23: 161627 3879048 java.util.HashMap$Entry </li></ul><ul><li>24: 231331 3701296 com.collaxa.cube.xml.dom.CubeDOMText </li></ul><ul><li>25: 50974 3262336 com.collaxa.cube.xml.dom.persistence.DomMoniker </li></ul><ul><li>26: 18916 2796096 [Ljava.lang.String; </li></ul><ul><li>27: 76813 2458016 org.apache.xerces.dom.TextImpl </li></ul><ul><li>28: 16101 1932120 oracle.xml.parser.schema.XSDElement </li></ul><ul><li>29: 48088 1923520 java.util.HashMap </li></ul><ul><li>30: 12178 1461360 oracle.dms.instrument.State </li></ul><ul><li>31: 12768 1456000 [Ljava.util.Hashtable$Entry; </li></ul><ul><li>32: 35861 1434440 oracle.dms.spy.Metric </li></ul><ul><li>33: 55646 1335504 java.util.Hashtable$Entry </li></ul><ul><li>34: 40418 1293376 oracle.xml.parser.v2.XMLText </li></ul>12096 126 [Loracle.toplink.internal.helper.IdentityHashtableEntry;: 12816 178 oracle.xml.parser.v2.XSLExprItem: 13184 412 java.lang.ref.Finalizer: 17472 42 [Lorg.apache.xerces.util.SymbolHash$Entry;: 18480 770 org.apache.xerces.util.SymbolTable$Entry: 19488 812 [Lorg.apache.xerces.impl.xs.identity.IdentityConstraint;: 20640 23 <constantPoolKlass>: 21568 674 oracle.jsp.parse.LineInfoMapObj: 27136 848 org.apache.xerces.impl.xs.XSParticleDecl: 39584 1237 oracle.xml.parser.v2.XMLElement: 43704 587 <symbolKlass>: 45472 812 org.apache.xerces.impl.xs.XSElementDecl: 51992 26 [I: 54704 -1 [Ljava.util.HashMap$Entry;: 56904 196 <methodKlass>: 84776 -18261 [C: 124080 7755 com.collaxa.cube.xml.dom.CubeDOMText: 3874872 161453 com.collaxa.cube.xml.dom.CubeDOMAttribute: 12266720 153334 com.collaxa.cube.xml.dom.CubeDOMElement: 15298056 637419 java.util.ArrayList: 303625912 718 [B: delta Bytes delta Instances Class Name
  9. 9. JADE Heap Analysis
  10. 10. JADE Heap Comparison
  11. 11. Analyzing Response Time Breakdown <ul><li>Key to optimization: where is response time coming from? </li></ul><ul><li>10.1.3 out-of-the-box capabilities </li></ul><ul><ul><li>BPEL Console – instance audit trails, statistics page </li></ul></ul><ul><ul><li>ESB Control – instances view processing time statistics </li></ul></ul><ul><ul><li>OWSM Monitor </li></ul></ul><ul><ul><li>EM transaction tracing </li></ul></ul><ul><li>Aggregating measurements </li></ul><ul><li>A single interaction’s invocation tree </li></ul><ul><li>Creative capabilities </li></ul>
  12. 12. BPEL Console Statistics Page
  13. 13. ESB Control Instances View Statistics
  14. 14. OWSM Monitor See Chapter 6 of OWSM Administrator’s Guide
  15. 15. EM Transaction Tracing
  16. 16. Analyzing Response Time Breakdown <ul><li>Key to optimization: where is response time coming from? </li></ul><ul><li>10.1.3 out-of-the-box capabilities </li></ul><ul><ul><li>BPEL Console – instance audit trails, statistics page </li></ul></ul><ul><ul><li>ESB Control – instances view processing time statistics </li></ul></ul><ul><ul><li>OWSM Monitor </li></ul></ul><ul><ul><li>EM transaction tracing </li></ul></ul><ul><li>Aggregating measurements </li></ul><ul><li>A single interaction’s invocation tree </li></ul><ul><li>Creative capabilities </li></ul>
  17. 17. Example Response Time Analysis Where is the Time Coming From? <ul><li>select domain, process, count(*), round(avg(time_taken),2) “Avg Time“ </li></ul><ul><li>from bpel_access </li></ul><ul><li>group by domain, process </li></ul><ul><li>order by 3 desc </li></ul><ul><li>Domain Process #Txns Avg Time </li></ul><ul><li>Domain_T Process_A 22161 23.53 </li></ul><ul><li>Domain_R Process_A 6865 7.37 </li></ul><ul><li>Domain_V Process_A 760 5.74 </li></ul><ul><li>Domain_N Process_A 439 4.43 </li></ul><ul><li>Domain_B Process_A 227 7.78 </li></ul><ul><li>Domain_T Process_R 51 2.73 </li></ul><ul><li>Domain_V Process_R 36 9.69 </li></ul><ul><li>Domain_S Process_A 35 6.51 </li></ul><ul><li>Domain_T Process_C 13 14.77 </li></ul><ul><li>Domain_B Process_D 12 4.17 </li></ul><ul><li>Domain_N Process_R 9 0.67 </li></ul><ul><li>Domain_B Process_R 5 7.4 </li></ul>
  18. 18. Example Response Time Analysis From Monitoring to Analysis <ul><li>We now knew what was slow, and how slow </li></ul><ul><li>Began evaluating hypotheses on why it was slow </li></ul><ul><li>Applied BPEL performance tuning playbook </li></ul><ul><ul><li>Thread counts </li></ul></ul><ul><ul><li>Durable -> transient </li></ul></ul><ul><ul><li>Minimized logging </li></ul></ul><ul><ul><li>Deferred dehydration </li></ul></ul><ul><ul><li>Removed BAM sensors </li></ul></ul><ul><li>Used monitoring to observe effects </li></ul><ul><li>None of that solved the problem </li></ul><ul><li>Suspected endpoint as major contributor </li></ul>
  19. 19. Example Response Time Analysis Correlated with Load – Endpoint Overloaded?
  20. 20. Creative Capabilities Embed Timing Instrumentation in BPEL Process <ul><li><bpelx:exec name=&quot;assign_invoke_start_time&quot; language=&quot;Java&quot; version=&quot;1.4&quot;><![CDATA[setVariableData(&quot;invoke_start_time&quot;, new Long(System.currentTimeMillis()));]]/> </li></ul><ul><li><invoke name=&quot;InvokeEndpoint&quot; partnerLink=“EndpointLink&quot; portType=&quot;ns3:Endpoint&quot; operation=“A&quot; inputVariable=“EndpointRequest&quot; outputVariable=“EndpointResponse&quot;/> </li></ul><ul><li><bpelx:exec name=&quot;output_invoke_response_time&quot; language=&quot;Java&quot; version=&quot;1.4&quot;><![CDATA[long endTime = System.currentTimeMillis(); </li></ul><ul><li>long startTime = ((Long) getVariableData(&quot;invoke_start_time&quot;)).longValue(); </li></ul><ul><li>long responseTime = endTime - startTime; </li></ul><ul><li>StringBuffer stringBuffer = new StringBuffer(); </li></ul><ul><li>stringBuffer.append(&quot;TIMING,&quot;); </li></ul><ul><li>stringBuffer.append(getVariableData(&quot;domain_id&quot;)); </li></ul><ul><li>stringBuffer.append(&quot;,&quot;); </li></ul><ul><li>stringBuffer.append(getVariableData(&quot;process_id&quot;)); </li></ul><ul><li>stringBuffer.append(&quot;,&quot;); </li></ul><ul><li>stringBuffer.append(getVariableData(&quot;instance_id&quot;)); </li></ul><ul><li>stringBuffer.append(&quot;,InvokeEndpoint,&quot;); </li></ul><ul><li>stringBuffer.append(responseTime); </li></ul><ul><li>System.out.println(stringBuffer.toString());]]/> </li></ul>
  21. 21. Creative Capabilities Embed Timing Instrumentation in BPEL Process <ul><li>Yields the following output in BPEL JVM output stream: </li></ul><ul><li>07/04/12 15:11:55 TIMING,Domain_T,Process_A,7192844,InvokeEndpoint,171933 </li></ul><ul><li>Added another table to the monitoring schema, and Java loader </li></ul><ul><li>create table ENDPOINT_INVOCATION ( </li></ul><ul><li>NODE_ID VARCHAR2(16) NOT NULL, </li></ul><ul><li>TIMESTAMP TIMESTAMP WITH TIME ZONE NOT NULL, </li></ul><ul><li>DOMAIN VARCHAR2(64) NOT NULL, </li></ul><ul><li>PROCESS VARCHAR2(64) NOT NULL, </li></ul><ul><li>INSTANCE_ID NUMBER(38) NOT NULL, </li></ul><ul><li>ACTIVITY VARCHAR2(16) NOT NULL, </li></ul><ul><li>TIME_TAKEN NUMBER(38) NOT NULL </li></ul><ul><li>); </li></ul>
  22. 22. Example Response Time Analysis Endpoint Response Time Distribution
  23. 23. Example Response Time Analysis Root Cause Identification <ul><li>Endpoint vendor’s measurements differed from ours </li></ul><ul><li>Led us to focus on “the network” </li></ul><ul><li>tcpdumps revealed endpoint requests sent one-at-a-time </li></ul><ul><li>Close inspection showed </li></ul><ul><ul><li>Synchronization in third-party library used by HTTP adapter </li></ul></ul><ul><ul><li>Classic queuing behavior ensued under sufficient load </li></ul></ul><ul><ul><li>We could work around with adapter config parameters – lucky! </li></ul></ul><ul><li>Lessons learned </li></ul><ul><ul><li>Could have identified cause quickly with JADE thread analysis </li></ul></ul><ul><ul><li>Embedded instrumentation is crude, query dehydration store instead </li></ul></ul><ul><ul><li>Too many threads exhausts native stack space, gives OOME </li></ul></ul>
  24. 24. <Insert Picture Here> Performance Optimization <ul><li>Overview </li></ul><ul><li>Examples </li></ul><ul><li>Principles, Patterns, Playbooks </li></ul>
  25. 25. Performance Optimization <ul><li>There is no silver bullet one-size-fits-all prescription </li></ul><ul><li>Every situation is highly context-dependent </li></ul><ul><li>But there are some principles and patterns </li></ul><ul><li>And product tuning playbooks </li></ul><ul><li>With occasional exceptions, application architecture is a MUCH more significant determinant of performance than product configuration/tuning </li></ul>
  26. 26. BPEL Application Architecture NAÏVE! BPEL Host BPEL Server Must be product defect!
  27. 27. BPEL Application Architecture Clustering & Load Balancing for Scalability BPEL Host BPEL Server BPEL Host BPEL Server BPEL Host BPEL Server <ul><li>Clustering has implications for design & administration </li></ul><ul><li>See http://www.oracle.com/technology/products/ias/hi_av/BPEL_HA_Paper.pdf </li></ul>
  28. 28. BPEL Application Architecture Pattern: Tier per Pipeline Stage BPEL Host BPEL Server BPEL Host BPEL Server BPEL Host BPEL Server BPEL Host BPEL Server BPEL Host BPEL Server
  29. 29. Principles – Response Time <ul><li>Minimize inter-process communication </li></ul><ul><ul><li>Co-location not distribution </li></ul></ul><ul><ul><li>Granularity not chattiness </li></ul></ul><ul><li>Avoid marshalling </li></ul><ul><ul><li>Co-locate components </li></ul></ul><ul><ul><li>Configure optimizations </li></ul></ul><ul><li>Leverage caching </li></ul><ul><ul><li>Incorporate Coherence </li></ul></ul><ul><li>Move processing to data </li></ul><ul><ul><li>Logic in PL/SQL </li></ul></ul><ul><ul><li>XTP with Coherence </li></ul></ul>
  30. 30. Principles - Throughput <ul><li>Network of queues model, queuing theory </li></ul><ul><li>Queues build upstream of slow service / great constraint </li></ul><ul><li>Theory of constraints – remove constraints </li></ul><ul><ul><li>Open throttles as much as is beneficial and reasonable </li></ul></ul><ul><ul><li>Eventually a resource will be exhausted </li></ul></ul><ul><ul><ul><li>Heap space (manifesting as GC frequency) </li></ul></ul></ul><ul><ul><ul><li>Connections to resources </li></ul></ul></ul><ul><ul><ul><li>CPU cycles </li></ul></ul></ul><ul><ul><ul><li>Network bandwidth </li></ul></ul></ul>
  31. 31. Principles – Heap Utilization <ul><li>Reduce thread count – fewer threads to concurrently instantiate objects on heap </li></ul><ul><li>Reduce caching </li></ul><ul><li>Reduce the size of the object graphs the application works with at a time, if possible </li></ul><ul><ul><li>Stream the input (batching) </li></ul></ul><ul><ul><li>In ORM-based services: more querying, less caching </li></ul></ul>
  32. 32. Patterns <ul><li>SOA patterns is a nascent field – choose wisely </li></ul><ul><ul><li>Start with Gregor Hohpe’s survey </li></ul></ul><ul><ul><ul><li>http://eaipatterns.com/ramblings/52_soapatterns.html </li></ul></ul></ul><ul><li>Grid patterns is even more nascent </li></ul><ul><ul><li>Choose even more wisely </li></ul></ul><ul><li>ORM performance optimization patterns </li></ul><ul><ul><li>Replace Caching Variable with Query (for heap, response time relief) </li></ul></ul><ul><ul><li>Repository plus Query Object (for ripple load mitigation) </li></ul></ul><ul><li>Performance engineering patterns </li></ul><ul><ul><li>Invocation Tree </li></ul></ul><ul><ul><li>Production Replay </li></ul></ul>
  33. 33. Product Tuning Playbooks <ul><li>BPEL </li></ul><ul><ul><li>OOW 2006 BPEL performance presentation (S282687) </li></ul></ul><ul><ul><li>BPEL performance webinar on OTN </li></ul></ul><ul><li>ESB </li></ul><ul><ul><li>ESB performance document on OTN </li></ul></ul>
  34. 34. <Insert Picture Here> Summary, Q&A
  35. 35. Summary <ul><li>APM is a professional responsibility of app ops/dev teams </li></ul><ul><li>Adopt tooling for monitoring, publish reports regularly </li></ul><ul><ul><li>Analyses often need highly purpose-specific, custom charts </li></ul></ul><ul><li>Use scientific method for analysis </li></ul><ul><li>Optimization is an art form </li></ul><ul><ul><li>Architectural principles and patterns </li></ul></ul><ul><ul><li>Product tuning playbooks </li></ul></ul><ul><li>Engineering means specifying, testing before production! </li></ul><ul><ul><li>Best practices for performance requirements specification, and load testing, are whole other topics unto themselves! </li></ul></ul>
  36. 36. Summary Prospects for Standarization Collect, store, and chart standard measures Yes Montoring At a high level, it’s a development lifecycle activity, with best practices Maybe Engineering Can have a standard APM program of monitoring, analysis, optimization Yes Management Patterns and playbooks apply here, but there is context-sensitivity & art Probably not Optimization Maybe Standardizable? Use the scientific method. It’s often a matter of understanding the response time breakdown or the throughput bottleneck. Analysis Remarks Activity
  37. 37. A Q & Q U E S T I O N S A N S W E R S
  38. 38. For More Information search.oracle.com +&quot;application performance management&quot; +&quot;service-oriented architecture&quot;

×