Oracle SOA Suite Performance Tuning- UKOUG Application Server & Middleware SIG Meeting

6,320 views

Published on

Matt Brasier, C2B2 Head of Consulting speaking at the UK Oracle User Group App Server & Middleware Special Interest Group Event on Wednesday, the 9th of October 2013.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
6,320
On SlideShare
0
From Embeds
0
Number of Embeds
4,107
Actions
Shares
0
Downloads
104
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Oracle SOA Suite Performance Tuning- UKOUG Application Server & Middleware SIG Meeting

  1. 1. © C2B2 Consulting Limited 2013 All Rights Reserved Oracle SOA Suite 11g Performance Tuning Matt Brasier C2B2 Consulting
  2. 2. © C2B2 Consulting Limited 2013 All Rights Reserved Who am I? • Head of consulting at C2B2 • 12 years experience with application servers – WebLogic 5.1 to present day • Consulting with C2B2 since 2006 – BEA Professional services – Red Hat Global Services – SME start ups to UK Govt • Author of SOA Suite 11g performance tuning cookbook
  3. 3. © C2B2 Consulting Limited 2013 All Rights Reserved Agenda • Performance and SOA Suite • A performance tuning methodology for SOA Suite • Common SOA Suite performance problems • Questions
  4. 4. © C2B2 Consulting Limited 2013 All Rights Reserved PERFORMANCE AND SOA SUITE Key challenges in SOA Suite performance
  5. 5. © C2B2 Consulting Limited 2013 All Rights Reserved Key Challenges • Fundamental to many organisations SOA infrastructure – It needs to work invisibly and quietly – Plumbing • Hard to test – Requirements vary significantly between applications • Limited documentation
  6. 6. © C2B2 Consulting Limited 2013 All Rights Reserved Key Challenges • Different components have different uses – Different customers use different components – There is no single answer to tuning such a wide range of products • Technologies – BPM – BPEL – Mediator – BAM – OSB
  7. 7. © C2B2 Consulting Limited 2013 All Rights Reserved Fundamental Nature • A message processing pipeline – Data size and quality has a large impact on performance • Frequent small messages vs infrequent large ones • A Java application – Subject to garbage collection and other JVM features
  8. 8. © C2B2 Consulting Limited 2013 All Rights Reserved Goals • Out of the box, Oracle SOA Suite is tuned for developer productivity. • Tune to remove bottlenecks – Performance tuning where there is no bottleneck has no result • Maximise performance and availability with existing infrastructure
  9. 9. © C2B2 Consulting Limited 2013 All Rights Reserved A PERFORMANCE TUNING METHODOLOGY FOR SOA SUITE How do we do this?
  10. 10. © C2B2 Consulting Limited 2013 All Rights Reserved Methodolody • Treat it the same as any other Java application
  11. 11. © C2B2 Consulting Limited 2013 All Rights Reserved A Science and an Art Test Identify Bottleneck Identify Cause Eliminate Bottleneck
  12. 12. © C2B2 Consulting Limited 2013 All Rights Reserved Testing • Use a proper test harness • Design users tests that accurately simulate user behaviour – All of it, at once – 10000 users logging in, then out of a site, over and over is not representative of your load • Use a sufficiently powerful load generation capability – Don’t test the performance of your load generator
  13. 13. © C2B2 Consulting Limited 2013 All Rights Reserved Monitoring • Goes hand-in-hand with testing • Testing is not a pass/fail activity – How close was the system to failing? – What happens if I double the load? – Which components are likely to fail first? • Do not view your system as a black box
  14. 14. © C2B2 Consulting Limited 2013 All Rights Reserved Monitoring • A JMX based monitoring framework is essential – Memory stats (inc GC) – Connection pool stats – JMS queue stats – Transaction stats – Data • Not just CPU% and Memory%
  15. 15. © C2B2 Consulting Limited 2013 All Rights Reserved Identify Bottleneck • Use the monitoring as a guide • What resource is constrained? – CPU – Memory – Locks – Bandwidth – Storage – Resource pools
  16. 16. © C2B2 Consulting Limited 2013 All Rights Reserved Identify Cause • Now you know what is bottlenecked, but why? – Bad code – Sub-optimal setting – Data overload – Data structure – Insufficient resources
  17. 17. © C2B2 Consulting Limited 2013 All Rights Reserved Useful tools • WebLogic console • Enterprise Manager • JStack • JStat • JVisualVM • JRockit Mission Control (JRMC)
  18. 18. © C2B2 Consulting Limited 2013 All Rights Reserved Identify Solution • The solution depends on the cause – Configuration changes – Code changes – More hardware!* • * Beware anyone who starts out with this assumption
  19. 19. © C2B2 Consulting Limited 2013 All Rights Reserved Implement solution • Implement the solution – In an environment where the problem could be reproduced • Test again – Has the bottleneck gone away? – Is there a new bottleneck?
  20. 20. © C2B2 Consulting Limited 2013 All Rights Reserved Applying the methodology • Bottom up – Work from the CPU upwards to find bottleneck • Treat SOA Suite as any other Java application at this stage – Key suspects such as Garbage Collection and locking • Stop when you are meeting your targets – You can always go faster • Subject to the speed of light in copper cable
  21. 21. © C2B2 Consulting Limited 2013 All Rights Reserved COMMON SOA SUITE PERFORMANCE PROBLEMS Where to start?
  22. 22. © C2B2 Consulting Limited 2013 All Rights Reserved Common Problems • Excessive garbage collection • Thread pools • Database access
  23. 23. © C2B2 Consulting Limited 2013 All Rights Reserved Garbage Collection • SOA Suite applications vary in how much data they have to process and how fast
  24. 24. © C2B2 Consulting Limited 2013 All Rights Reserved • However it is often the case that they process lots of XML – XML is object creation intensive – Results in garbage • Garbage collection is based on the JVM – Hotspot/JRockit
  25. 25. © C2B2 Consulting Limited 2013 All Rights Reserved Garbage Collection • Use tools like visualvm, jrmc or jstat to identify garbage collection problems – Garbage collection pause times of > 1s per minute • Tune the garbage collector – Parallel is a good starting point – Reduce the number and duration of full GC pauses • Tune the memory pools – Generational memory
  26. 26. © C2B2 Consulting Limited 2013 All Rights Reserved Tools • Jstat – Jstat –gccause <pid> • JVisualVM
  27. 27. © C2B2 Consulting Limited 2013 All Rights Reserved Thread Pools • There are a lot of thread pools – Use a tool to see which thread pools are exhusted • JStack • JVisualVM • Etc • WebLogic thread pool
  28. 28. © C2B2 Consulting Limited 2013 All Rights Reserved Thread Pools • BPEL/BPMN thread pools – Dispatcher Invoke threads • Process requests from external sources – Dispatcher engine threads • Process asynchronous callback responses – Dispatcher System threads • Housekeeping tasks
  29. 29. © C2B2 Consulting Limited 2013 All Rights Reserved Database access • Oracle SOA Suite makes heavy use of the database – Persistence of data – Metadata – Auditing and logging
  30. 30. © C2B2 Consulting Limited 2013 All Rights Reserved Database access • Database access can be exponential with load – Each new user adds persistence load, auditing load, metadata lookups etc. • Reduce the amount of database access to improve performance – Reduce auditing – Reduce persistence of in-flight processes
  31. 31. © C2B2 Consulting Limited 2013 All Rights Reserved Database access • Reduce persistence – InMemoryOptimisation (transient processes) – Reduce audit threshold – Reduce completion persist level – Increase audit trail size threshold – Increase large document threshold – Many many more
  32. 32. © C2B2 Consulting Limited 2013 All Rights Reserved Database access • Tune the database – Purge old BPEL processes – General database tuning
  33. 33. © C2B2 Consulting Limited 2013 All Rights Reserved Summary • Treat SOA Suite as any other java EE application – One that processes a lot of XML • Reduce database access • Manage memory correctly • Tune thread pools
  34. 34. © C2B2 Consulting Limited 2013 All Rights Reserved FOCUS TUNING ONLY WHERE IT IS REQUIRED You can’t make it faster if it is not a bottleneck
  35. 35. © C2B2 Consulting Limited 2013 All Rights Reserved Questions? • Thankyou for your time mbrasier@c2b2.co.uk @c2b2consulting, @mbrasier, www.c2b2.co.uk

×