Debugging java deployments_2

3,532 views
3,395 views

Published on

JavaOne 2011 Server production JVM Debugging talk

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,532
On SlideShare
0
From Embeds
0
Number of Embeds
1,358
Actions
Shares
0
Downloads
103
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide
  • Session ID 22723 Status Accepted Title JVM Flight Simulator: Debugging Java Deployments Abstract Troubleshooting issues such as instances of OutOfMemoryError, performance problems, and various exceptions is a common task for anyone developing or deploying an application. This deep dive session presents a hands-on demo of using open source IBM tools such as Monitoring and Diagnostic Tools for Java, Extended Memory Analyzer Tool, and the Support Assistant. Come learn how to diagnose these common problem types. Speakers Rohit Kelapure IBM Advisory Software EngineerType Conference Session Length 60 minutes JavaOne Primary Track Core Java Platform JavaOne Optional Track Java SE, Client Side Technologies, and Rich User Experiences
  • Challenge #5: Threading and Synchronization Issues Of the many issues affecting the performance of Java applications, synchronization ranks near the top. There is no question that synchronization is necessary to protect shared data in an application. The fundamental need to synchronize lies with Java's support for concurrency. This happens by allowing the execution of code by separate threads within the same process. Using shared resources efficiently, such as connection pools and data caches, is very important for good application performance. Being too aggressive with shared resources causes data inconsistencies, and being too conservative leads to too much contention between threads (because resource locking is involved). This affects the performance of the application largely because most threads servicing users are affected and slowed down -- they end up waiting for resources instead of doing real processing work.If you want to improve synchronization issues, application performance management tools can help; the right tool can enable you to monitor application execution under high loads (aka "in production") and quickly pinpoint the execution times. In doing so, you will increase your ability to identify thread synchronization issues become greatly increase -- and the overall MTTR will drop dramatically.
  • Length of time in seconds thread can be active before considered hung Number of times that false alarms can occur before automatically increasing the thresholdOpportunity to implement in apache Commons ThreadPoolDoes not include any spawned threads or unmanaged threads
  • Calling interrupt does not necessarily stop the target thread from doing what it is doing; it merely delivers the message that interruption has been requestedDeprecated in JDK1.2 because it can corrupt object state:General‐purpose application task & application library code should never swallow interruption requests
  • and makes enough progress that you can get its attention some other way
  • Path to GC roots – Reference chain that prevents object from being GcedDominator tree grouped by Class Loader- Limit analysis to a single application in a JEE environment Retained Size Graphs- set of objects that can be reclaimed if we could delete XSELECT data as "MemorySessionData", data.@usedHeapSize as "Heap Size", data.@retainedHeapSize as "Retained Heap Size", mSwappableData, toString(data.mManager.scAppParms._J2EEName) as "J2EE Name", toString(data.appName) as "App Name", toString(data.mSessionId) as "Session ID" FROM com.ibm.ws.webcontainer.httpsession.MemorySessionData data
  • If the Javacores show most threads are idle, it is possible that the requests are not making their way to the Application ServerThe following example shows multiple threads waiting for a DB2 database to respond. This indicates the bottleneck is in the DB
  • Connection Manager, Node Synchronization, Node agent, Deployment Manager, WebContainer Runtime AdvisorSpecialized tracing and Runtime checksConnection Leak, WsByteBuffer leak detection, Session crossover, transaction ID, request ID The advisors provide a variety of advice on the following application server resources: Object Request Broker service thread pools Web container thread pools Connection pool size Persisted session size and time Data source statement cache size Session cache size Dynamic cache size Java virtual machine heap size DB2 Performance Configuration wizard
  • Debugging java deployments_2

    1. 1. Rohit Kelapure<br />IBM Advisory Software Engineer <br />29 September 2011<br />Server Resiliency - Debugging Java deployments<br />
    2. 2. Introduction to Speaker – Rohit Kelapure<br />Responsible for the resiliency of WebSphere Application Server<br />Team Lead and architect of Caching & Data replication features in WebSphere<br />Called upon to hose down fires & resolve critical situations<br />Customer advocate for large banks <br />Active blogger All Things WebSphere<br />Apache Open Web Beans committer <br />Java EE, OSGI & Spring Developer<br />kelapure@us.ibm.com<br />kelapure@gmail.com<br />Linkedin<br />http://twitter.com/#!/rkela<br />2<br />
    3. 3. Important Disclaimers<br />THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. <br />WHILST EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. <br />ALL PERFORMANCE DATA INCLUDED IN THIS PRESENTATION HAVE BEEN GATHERED IN A CONTROLLED ENVIRONMENT. YOUR OWN TEST RESULTS MAY VARY BASED ON HARDWARE, SOFTWARE OR INFRASTRUCTURE DIFFERENCES.<br />ALL DATA INCLUDED IN THIS PRESENTATION ARE MEANT TO BE USED ONLY AS A GUIDE.<br />IN ADDITION, THE INFORMATION CONTAINED IN THIS PRESENTATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM, WITHOUT NOTICE. <br />IBM AND ITS AFFILIATED COMPANIES SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION. <br />NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE EFFECT OF: <br />- CREATING ANY WARRANT OR REPRESENTATION FROM IBM, ITS AFFILIATED COMPANIES OR ITS OR THEIR SUPPLIERS AND/OR LICENSORS<br />3<br />
    4. 4. Copyright and Trademarks<br />© IBM Corporation 2011. All Rights Reserved. <br />IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corp., and registered in many jurisdictions worldwide. <br />Other product and service names might be trademarks of IBM or other companies. <br />A current list of IBM trademarks is available on the Web – see the IBM “Copyright and trademark information” page at URL: www.ibm.com/legal/copytrade.shtml<br />4<br />
    5. 5. Outline<br />Server Resiliency Fundamentals<br />Common JVM Problems<br />Protecting your JVM<br />Hung thread detection, Thread Interruption, Thread hang recovery<br />Memory leak detection, protection & action<br />Scenario based problem resolution<br />Tooling<br />Eclipse Memory Analyzer<br />Thread Dump Analyzer<br />Garbage Collection and Memory Visualizer<br />5<br />
    6. 6. Resiliency<br />Property of a material that can absorb external energy when it is forced to deform elastically, and then be able to recover to its original form and release the energy<br />6<br />
    7. 7. Server Resiliency Concepts<br />7<br />October 4, 2011<br />Explicit Messaging<br />Distributed Shared memory – Better Consistency<br />Message Passing – Loose Coupling<br />Uniform Interface<br />E.g. World Wide Web<br />Better scalability, reusability and reliability <br />Data, process and other forms of computations identified by one mechanism <br />Semantics of operations in messages for operating on the data are unified<br />Self Management<br />Composed of self-managing components.<br />Managed element, managers, sensors & effectors<br />e.g. TCP/IP Congestion Control<br />Redundancy (Data and processing)<br />Create Replicas<br />High cost of initialization and reconfiguration<br />Redundant elements need to be synchronized from time to time<br />Partition<br />Splitting the data into smaller pieces and storing them in distributed fashion<br />Allows for parallelization & divide and conquer<br />Partial failure isolation<br />Virtualization<br />Functionalities of processing and data element virtualized as a service<br />Loose coupling between system and consumed services<br />Integration by enforcing explicitly boundary and schema-based interfaces<br />Decentralized Control <br />High communication overhead of centralized control for a system of heavy redundancy<br />Sometimes trapped in locally optimized solutions<br />Fixing issues requires shutting down the entire system e.g. AWS outage<br />
    8. 8. Most common JVM Problem Scenarios<br />8<br />October 4, 2011<br />
    9. 9. Thread Hangs<br />Threading and synchronization issues are among the top 5 application performance challenges<br />too aggressive with shared resources causes data inconsistencies<br />too conservative leads to too much contention between threads <br />Application unresponsiveness <br />Adding users / threads /CPUs causes app slow down (less throughput, worse response)<br />High lock acquire times & contention<br />Race conditions, deadlock, I/O under lock<br />Tooling is needed to rescue applications and the JVM from itself <br />Identify these conditions <br />If possible remedy them in the short term for server resiliency<br />9<br />October 4, 2011<br />
    10. 10. JVM Hung Thread Detection <br />Every X seconds an alarm thread wakes up and iterates over all managed thread pools. <br />Subtract the "start time" of the thread from the current time, and passes it to a monitor. <br />Detection policy then determines based on the available data if the thread is hung <br />Print stack trace of the hung thread<br />10<br />October 4, 2011<br />
    11. 11. Thread Interruption 101 <br />Thread.stop stops thread by throwing ThreadDeath exception * Deprecated<br />Thread.interrupt(): Cooperative mechanism for a thread to signal another thread that it should, at its convenience and if it feels like it, stop what it is doing and do something else.<br />Interruption is usually the most sensible way to implement task cancellation. <br />Because each thread has its own interruption policy, you should not interrupt a thread unless you know what interruption means to that thread.<br />Any method sensing interruption should<br />Assume current task is cancelled & perform some task‐specific cleanup<br />Exit as quickly and cleanly as possible ensuring that callers are aware of cancellation <br />Propagate the exception, making your method an interruptible blocking method, to throw new InterruptedException()<br />Restore the interruption status so that code higher up on the call stack can deal with it Thread.currentThread().interrupt()<br />Only code that implements a thread's interruption policy may swallow an interruption request. <br />11<br />October 4, 2011<br />
    12. 12. Interrupting threads<br />12<br />
    13. 13. 13<br />October 5, 2011<br />CancellingThreads <br />
    14. 14. Dealing with Non‐interruptible Blocking<br />Many blocking library methods respond to interruption by returning early and throwing InterruptedException<br />Makes it easier to build tasks that are responsive to cancellation <br />Lock.lockInterruptibly<br />Thread.sleep,<br />Thread.wait<br />Thread.notify<br />Thread.join<br />Not all blocking methods or blocking mechanisms are responsive to interruption<br />if a thread is blocked performing synchronous socket I/O, interruption has no effect other than setting the thread's interrupted status<br />If a thread is blocked waiting for an intrinsic lock, there is nothing you can do to stop short of ensuring that it eventually acquires the lock<br />14<br />October 4, 2011<br />
    15. 15. Thread Hang Recovery – Technique <br />Application specific hacks for thread hang recovery<br />Byte code instrumentation<br />Transform the concrete subclasses of the abstract classes InputStream& OutputStreamto make the socket I/O operations interruptible. <br />Transform an application class so that every loop can be interrupted by invoking Interrupter.interrupt(Thread, boolean)<br />Transform a monitorenter instruction and a monitorexit instruction so that the wait at entering into a monitor is interruptible<br />http://www.ibm.com/developerworks/websphere/downloads/hungthread.html<br />15<br />
    16. 16. Memory Leaks<br />Leaks come in various types, such as<br />Memory leaks<br />Thread and ThreadLocal leaks<br />ClassLoader leaks<br />System resource leaks<br /> Connection leaks<br />Customers want to increase application uptime without cycling the server. <br />Frequent application restarts without stopping the server.<br />Frequent redeployments of the application result in OOM errors <br />What do we have today <br />Offline post-mortem analysis of a JVM heap. Tools like Jrockit Mission Control, MAT. IEMA are the IBM Extensions for Memory Analyzer<br />Runtime memory leak detection using JVMTI and PMI (Runtime Performance Advisor)<br />We don’t have application level i.e. top down memory leak detection and protection<br />Leak detection by looking at suspect patterns in application code<br />16<br />October 4, 2011<br />
    17. 17. ClassLoader Leaks 101<br />A class is uniquely identified by<br />Its name + The class loader that loaded it<br />Class with the same name can be loaded multiple times in a single JVM, each in a different class loader<br />Web containers use this for isolating web applications<br />Each web application gets its own class loader<br />Reference Chain<br />An object retains a reference to the class it is an instance of<br />A class retains a reference to the class loader that loaded it<br />The class loader retains a reference to every class it loaded<br />Retaining a reference to a single object from a web application pins every class loaded by the web application<br />These references often remain after a web application reload With each reload, more classes get pinned ultimately leading to an OOM<br />17<br />October 4, 2011<br />
    18. 18. Tomcat pioneered approach - Leak Prevention <br />JRE triggered leak <br />Singleton / static initializer<br />Can be a Thread<br />Something that won’t get garbage collected<br />Retains a reference to the context class loader when loaded<br />If web application code triggers the initialization<br />The context class loader will be web application class loader<br />A reference is created to the web application class loader<br />This reference is never garbage collected<br />Pins the class loader (and hence all the classes it loaded) in memory<br />Prevention with a DeployedObjectListener<br />Calling various parts of the Java API that are known to retain a reference to the current context class loader<br />Initialize these singletons when the Application Server’s class loader is the context class loader<br />18<br />October 5, 2011<br />
    19. 19. Leak Detection<br />Application Triggered Leaks<br />ClassLoader <br />Threads<br />ThreadLocal <br />JDBC Drivers<br />Non Application<br />RMI Targets<br />Resource Bundle<br />Static final references<br />InstrospectionUtils<br />Loggers<br />Prevention<br />Code executes when a web application is stopped, un-deployed or reloaded<br />Check, via a combination of standard API calls and some reflection tricks, for known causes of memory leaks<br />19<br />October 4, 2011<br />
    20. 20. Memory leak detection console<br />20<br />October 4, 2011<br />
    21. 21. What is wrong with my application …?<br />Why does my application run slow every time I do X ?<br />Why does my application have erratic response times ? <br />Why am I getting Out of Memory Errors ?<br />What is my applications memory footprint ?<br />Which parts of my application are CPU intensive ?<br />How did my JVM vanish without a trace ?<br />Why is my application unresponsive ?<br />What monitoring do I put in place for my app. ?<br />21<br />October 4, 2011<br />
    22. 22. What is your JVM up to ?<br />Windows style task manager for displaying thread status and allow for their recovery & interruption<br />Leverage the ThreadMXBean API in the JDK to display thread information<br />https://github.com/kelapure/dynacache/blob/master/scripts/AllThreads.jsphttps://github.com/kelapure/dynacache/blob/master/scripts/ViewThread.jsp<br />22<br />October 4, 2011<br />
    23. 23. Application runs slow when I do XXX ?<br />Understand impact of activity on components<br />Look at the thread & method profiles <br />IBM Java Health Center <br />Visual VM<br />Jrockit Mission Control<br />JVM method & dump trace - pinpoint performance problems. <br />Shows entry & exit times of any Java method<br />Method to trace to file for all methods in tests.mytest.package<br />Allows taking javadump, heapdump, etc when a method is hit<br />Dump javacore when method testInnerMethod in an inner class TestInnerClass of a class TestClass is called<br />Use Btrace, -Xtrace * –Xdump to trigger dumps on a range of events<br />gpf, user, abort, fullgc, slow, allocation, thrstop, throw …<br />Stack traces, tool launching<br />23<br />October 4, 2011<br />
    24. 24. Application has erratic response times ?<br />Verbose gc should be enabled by default<br /><2% impact on performance<br />VisualGC, GCMV &PMAT : Visualize GC output <br />In use space after GC<br />Positive gradient over time indicates memory leak<br />Increased load (use for capacity plan) <br />Memory leak (take HDs for PD.)<br /> Choose the right GC policy <br />Optimized for “batch” type applications, consistent allocation profile<br />Tight responsiveness criteria, allocations of large objects<br />High rates of object “burn”, large # of transitional objects<br />12, 16 core SMP systems with allocation contention (AIX only)<br />GC overhead > 10%  wrong policy | more tuning<br />Enable compressed references for 64 bit JVM <br />24<br />October 5, 2011<br />
    25. 25. Out Of Memory Errors ?<br />JVM Heap sized incorrectly<br />GC adapts heap size to keep occupancy [40, 70]%<br />Determine heap occupancy of the app. under load<br />Xmx = 43% larger than max. occupancy of app.<br />For 700MB occupancy , 1000MB Max. heap is reqd. (700 +43% of 700)<br />Analyze heapdumps & system dumps with tools like Eclipse Memory Analyzer<br />Lack of Java heap or Native heap<br />Eclipse Memory Analyzer and IBM extensions <br />Finding which methods allocated large objects<br />Prints stacktrace for all objects above 1K<br />Enable Java Heap and Native heap monitoring <br />JMX and metrics output by JVM<br />Classloader exhaustion<br />25<br />October 4, 2011<br />
    26. 26. Applications memory footprint ?<br />HPROF – profiler shipped with JDK – uses JVMTI <br />Analysis of memory usage -Xrunhprof:heap=all <br />Performance Inspector tools - JPROF Java Profiling Agent<br />Capture state of the Java Heap later processed by HDUMP<br />Group a system dump by classloader<br />since each app has its own classloader, you can get accurate information on how much heap each application is taking up<br />Use MAT to investigate heapdumps & system dumps <br />Find large clumps, Inspect those objects, What retains them ?<br />Why is this object not being garbage collected – <br />List Objects > incoming refs, Path to GC roots, Immediate dominators <br />Limit analysis to a single application in a JEE environment - Dominator tree grouped by ClassLoader Dominator tree grouped by Class Loader<br />Set of objects that can be reclaimed if we could delete X - Retained Size Graphs Retained Size Graphs <br />Traditional memory hogs like HTTPSession, Cache - Use Object Query Language (OQL<br />Use Object Query Language (OQL)<br />26<br />October 4, 2011<br />
    27. 27. Using Javacores for Troubleshooting<br />Javacores are often the most critical piece of information to resolve a hang, high CPU, crash and sometimes memory problems<br />A Javacore is a text file that contains a lot of useful information<br />The date, time, java™ version, full command path and arguments<br />All the threads in the JVM, including thread state, priority, thread ID, name<br />Thread call stacks<br />Javacores can be generated automatically or on demand<br />Automatically when an OutOfMemoryException is thrown<br />On demand with “kill -3 <pid>” <br />Message to the SystemOut when a javacore is generated<br />27<br />"WebContainer : 537" (TID:0x088C7200, sys_thread_t:0x09C19F00, state:CW, native ID:0x000070E8) prio=5<br /> at java/net/SocketInputStream.socketRead0(Native Method)<br /> at java/net/SocketInputStream.read(SocketInputStream.java:155)<br /> at oracle/net/ns/Packet.receive(Bytecode PC:31)<br /> at oracle/net/ns/DataPacket.receive(Bytecode PC:1)<br /> at oracle/net/ns/NetInputStream.read(Bytecode PC:33)<br /> at oracle/jdbc/driver/T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1123)<br /> at oracle/jdbc/driver/T4C8Oall.receive(T4C8Oall.java:480)<br /> at oracle/jdbc/driver/T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:813)<br /> at oracle/jdbc/driver/OracleStatement.doExecuteWithTimeout(OracleStatement.java:1154)<br /> at oracle/jdbc/driver/OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3415)<br />at com/ibm/commerce/user/objects/EJSJDBCPersisterCMPDemographicsBean_2bcaa7a2.load()<br /> at com/ibm/ejs/container/ContainerManagedBeanO.load(ContainerManagedBeanO.java:1018)<br /> at com/ibm/ejs/container/EJSHome.activateBean(EJSHome.java:1718)<br />
    28. 28. CPU intensive parts of the app?<br />ThreadDumps or Javacores- Poor mans profiler<br />Periodic javacores<br />Thread analysis – using the Thread Monitor Dump Analyzer tool<br />High CPU is typically diagnosed by comparing two key pieces of information<br />Using Javacores, determine what code the threads are executing<br />Gather CPU usage statistics by thread<br />For each Javacore compare the call stacks between threads<br />Focus first on Request processing threads first<br />Are all the threads doing similar work? <br />Are the threads moving ?<br />Collect CPU statistics per thread<br />Is there one thread consuming most of the CPU?<br />Are there many active threads each consuming a small percentage of CPU?<br />High CPU due to excessive garbage collection ?<br />If this is a load/capacity problem then use HPROF profiler <br />-Xrunhrof:cpu=samples, -Xrunhprof:cpu=time<br />28<br />October 4, 2011<br />
    29. 29. Diagnosis - Hangs<br />Often hangs are due to unresponsive synchronous requests<br />SMTP Server, Database, Map Service, Store Locator, Inventory, Order processing, etc<br />3XMTHREADINFO "Servlet.Engine.Transports : 11" (TID:0x7DD38040, sys_thread_t:0x44618828, state:R, native ID:0x4A9F) prio=54XESTACKTRACE at COM.ibm.db2.jdbc.app.DB2PreparedStatement.SQLExecute()4XESTACKTRACE at COM.ibm.db2.jdbc.app.DB2PreparedStatement.execute2(DB2PreparedStatement.java)4XESTACKTRACE at COM.ibm.db2.jdbc.app.DB2PreparedStatement.executeQuery(DB2PreparedStatement.java()4XESTACKTRACE at ...<br />3XMTHREADINFO "Servlet.Engine.Transports : 12" (TID:0x7DD37FC0, sys_thread_t:0x4461BDA8, state:R, native ID:0x4BA0) prio=54XESTACKTRACE at COM.ibm.db2.jdbc.app.DB2PreparedStatement.SQLExecute()4XESTACKTRACE at COM.ibm.db2.jdbc.app.DB2PreparedStatement.execute2(DB2PreparedStatement.java)4XESTACKTRACE at COM.ibm.db2.jdbc.app.DB2PreparedStatement.executeQuery(DB2PreparedStatement.java()4XESTACKTRACE at ...<br />3XMTHREADINFO "Servlet.Engine.Transports : 13" (TID:0x7DD34C50, sys_thread_t:0x4465B028, state:R, native ID:0x4CCF) prio=54XESTACKTRACE at COM.ibm.db2.jdbc.app.DB2PreparedStatement.SQLExecute()4XESTACKTRACE at COM.ibm.db2.jdbc.app.DB2PreparedStatement.execute2(DB2PreparedStatement.java)4XESTACKTRACE at COM.ibm.db2.jdbc.app.DB2PreparedStatement.executeQuery(DB2PreparedStatement.java()<br />Not all hangs are waiting on an external resource<br />A JVM can hang due to a synchronization problem - One thread blocking several others<br />29<br />3XMTHREADINFO "Servlet.Engine.Transports : 11" (TID:0x7DD38040, sys_thread_t:0x44618828, state:R, native ID:0x4A9F) prio=53LKMONOBJECT com/ibm/ws/cache/Cache@0x65FB8788/0x65FB8794: owner "Default : DMN0" (0x355B48003LKWAITERQ Waiting to enter:3LKWAITER "WebContainer : 0" (0x3ACCD000)3LKWAITER "WebContainer : 1" (0x3ACCCB00)3LKWAITER "WebContainer : 2" (0x38D68300)3LKWAITER "WebContainer : 3" (0x38D68800)<br />
    30. 30. How did my JVM vanish without trace ?<br />JVM Process Crash Usual Suspects<br /> Bad JNI calls, Segmentation violations, Call Stack Overflow<br />Native memory leaks - Object allocation fails with sufficient space in the JVM heap<br />Unexpected OS exceptions (out of disk space, file handles), JIT failures<br />Monitor the OS process size <br />Runtime check of JVM memory allocations –<br />Xcheck:memory<br />Native memory usage - Create a core dump on an OOM<br />JNI code static analysis -Xcheck:jni (errors, warnings, advice)<br />GCMV provides scripts and graphing for native memory<br />Windows “perfmon“, Linux “ps” & AIX “svmon” <br />Find the last stack of native code executing on the thread during the crash<br />The signal info (1TISIGINFO) will show the Javacore was created due to a crash<br />Signal 11 (SIGSEGV) or GPF<br />30<br />October 4, 2011<br />
    31. 31. What do I monitor ?<br />31<br />October 4, 2011<br />
    32. 32. Top Malpractices<br />32<br />October 4, 2011<br />
    33. 33. Support Assistant Workbench to help with Problem Determination<br />33<br />October 4, 2011<br />
    34. 34. One stop shop for tools to analyze JVM issues<br />34<br />October 4, 2011<br />
    35. 35. Tools<br />35<br />October 4, 2011<br />
    36. 36. Runtime Serviceability aids<br />Troubleshooting panels in the administration console<br />Performance Monitoring Infrastructure metrics <br />Diagnostic Provider Mbeans<br />Dump Configuration, State and run self-test<br />Application Response Measurement/Request Metrics <br />Follow transaction end-to-end and find bottlenecks<br />Trace logs & First Failure Data Capture<br />Runtime Performance Advisors<br />Memory leak detection, session size, …<br />Specialized tracing and Runtime checks<br />Tomcat Classloader Leak Detection<br />Session crossover, Connection leak, ByteBuffer leak detection <br />Runaway CPU thread protection<br />36<br />October 4, 2011<br />
    37. 37. References <br />Java theory and practice: Dealing with InterruptedException<br />http://www.ibm.com/developerworks/java/library/j-jtp05236/index.html<br />Architectural design for resilience<br />http://dx.doi.org/10.1080/17517570903067751<br />IBM Support Assistant<br />http://www-01.ibm.com/software/support/isa/download.html<br />How Customers get into trouble<br />http://www-01.ibm.com/support/docview.wss?uid=swg27008359<br />37<br />
    38. 38. Q&A<br />Thank You<br />38<br />October 4, 2011<br />

    ×