WebSphere performance tuning toolkit workshop

1. WebSphere Application Server Performance Tuning Toolkit
Workshop with Daytrader
Statement: draft

2. Author : Sui Peng Fei (suipf@cn.ibm.com)
Version: 1.0
Date: Jan, 16th, 2012

3. Table of Contents
Getting Start.....................................................................................................................................4
Performance Tuning Toolkit Overview.......................................................................................................4
Prepare for the workshop..........................................................................................................................6
Set up WebSphere Application Server Topology........................................................................................6
Download the WebSphere Application Server Performance Tuning Toolkit(PTT)......................................6
Download the Apache JMeter...................................................................................................................6
Install the Daytrader Application...............................................................................................................6
Have a brief stress testing using the JMeter Tool.....................................................................................10
Scenario1: “Synchronization Blocking” Problem............................................................................13
Determine the “Synchronization Blocking” problem...............................................................................16
Generate a thread dump.........................................................................................................................16
Launch the ISA 5.0 Tech Preview .............................................................................................................17
Use ISA V5 to analyze the “Synchronization Blocking” problem..............................................................18
Scenario2: “Dead Lock” problem..................................................................................................22
Monitoring the “Dead Lock” problem......................................................................................................23
Determine the “Dead Lock” problem.......................................................................................................25
Generate a thread dump.........................................................................................................................25
Use ISA V5 to analyze the “Dead Lock” problem......................................................................................26
Scenario3: “High CPU Usage” problem.........................................................................................30
Trigger the “High CPU Usage” problem....................................................................................................30
Monitoring the “High CPU Usage” problem.............................................................................................31
Determine the “High CPU Usage” problem..............................................................................................33
Scenario4: “Connection Leak” problem........................................................................................35
Application Changes to Simulate “Connection Leak” Situation................................................................35
Trigger the “Connection Leak” problem...................................................................................................36
Monitoring the “Connection Leak” problem............................................................................................37
Determine the “Connection Leak” problem.............................................................................................39
Scenario5: “Memory Leak” problem.............................................................................................42
Application Changes to Simulate “Memory Leak” Situation....................................................................42
Trigger the “Memory Leak” problem.......................................................................................................42
Monitoring the “Memory Leak” problem................................................................................................43
Determine the “Memory Leak” problem.................................................................................................44
Other optional functions you can go through................................................................................49
Tuning the performance related parameters...........................................................................................49
Adjust rules from rule editor if needed....................................................................................................50
Generate report.......................................................................................................................................52

4. Getting Start
Performance Tuning Toolkit Overview
The WebSphere® Application Server Performance Tuning Toolkit (PTT) is an
Eclipse-based intelligent tool designed to help users tune the performance of
WebSphere Application Server using data analysis, and statistical inference
technology. It is designed to help users locate bottlenecks and tune their applications
appropriately.
Compared with other performance tuning tools, the main character of PTT is light
weight and easy to use. You need not do any install or configuration work except
providing the IP and soap port of your Dmgr.
Below is the function list of the toolkit:
Functions Components
Performance  Dashboard (key status of all servers)
Monitoring
 Monitor page (chart of components of one server)
 Data page (detailed page of components)
Performance Tuning  “Tuning Parameters” view (tune common server
parameters)
 Scripts view (tune other parameters)
Health Check  Rule engine (process data)
 Rule editor (edit rules)
Operations  Dump agent (generate thread dump and heap
dump)
 Trace agent (enable trace)
 Connection agent (extract the connection pool
contents)
Generate Report  Report engine (generate, export and print report)
In order to help users learn how to use this toolkit, it provides a modified Daytrader
benchmark and a set of scripts to illustrate existing common customer issues and
solutions, thus greatly reducing the user’s learning curve. Through execution of the
workshop, you will have a good understanding of the impact of typical bad code, and

5. know more about symptoms of certain problems thereby speeding the process of
problem resolution.
The built-in Daytrader application
DayTrader is a open source benchmark built on a core set of Java EE technologies
that includes Java Servlets and JavaServer Pages (JSPs) for the presentation layer and
Java database connectivity (JDBC), Java Message Service (JMS), Enterprise
JavaBeans (EJBs) and Message-Driven Beans (MDBs) for the back-end business
logic and persistence layer. The following diagram provides a high-level overview of
the application architecture.
Figure 1: Application architecture
The built-in Daytrader application will simulate improper application design or some
error coding scenario which will involve "Performance" issues.
The simulated "sick" situations include:
1. Causal factors lead to synchronization blocking or lock competition, and the
threads have to spend more time on waiting for the lock.
2. Causal factors lead to a deadlock and block other threads which need to gain
the lock owned by deadlock threads.
3. Causal factors lead to heavy JDBC connection consuming, which will cause
lots of connection waiting threads.
4. High CPU usage cause by some CPU consuming threads.
5. Out of memory problems caused by memory leak.

6. Prepare for the workshop
 Set up WebSphere Application Server Topology
Install the WebSphere Application Server 7.0 or later and create a cluster with at least
2 members (Download the trial version from DeveloperWorks if needed). Create a
proxy server as the dispatcher.
 Download the WebSphere Application Server Performance Tuning
Toolkit(PTT)
Download link (Internal):
http://cattail.boulder.ibm.com/cattail/#view=suipf@cn.ibm.com/files/2F87C2500E94
3DDC86A2F0EC093F23B6
 Download the Apache JMeter
Download link: http://jmeter.apache.org/download_jmeter.cgi
Download the ISA 5.0 Tech Preview from IBM website.
Download link: http://www-01.ibm.com/support/docview.wss?uid=swg27023689
Install the Daytrader Application
Unzip the PTT zip file and double click the “PerfTuningToolkit.exe” to launch the
PTT workbench

7. Figure 2: PTT workbench
Click the “Add host” button in the toolbar of “Host” view to add a new host. Just
input the IP address and soap port of Dmgr. You need also provide the user name and
password if the global security of WAS is enabled.
Figure 3: Add the host info
Click the “Help” -> “Case Study” from the workbench menu bar to import the sample
application and script files to the “scripts” view.

8. Open the settings.py in the “Sample” folder and change the cluster name to the cluster
you created in previous steps. You also need to set the security settings if the WAS
security is enable.
Figure 4: Import the assets and modify settings
Select the host of your Dmgr in the host view, and right click the installDaytrader.py
in the sample folder of the “scripts” view. Select “run script” menu in the context
menu to run the installDaytrader.py script against your cell. The script will install the
modified Daytrader Application to the cluster you specified.

9. Figure 5: Install the Daytrader Application
Figure 6: Check the results of installation
Important: (Re)start the cluster and proxy server, and manually check the installed
application from url http://<proxy server host name>:<port>/daytrader/scenario

10. Figure 7: Verify the installation of Daytrader application
Note: the Daytrader application will use an embedded Derby data base, it is not a
valid design for a normal clustered application, but is very suitable for a demo
purpose.
You can also use DB2 as a central db for performance related testing, but you need to
specified the db2 settings in the settings.py and also need to created table and
populate data according to the instruction on the “configuration” tab of Daytrader
applications.
Have a brief stress testing using the JMeter Tool
Launch the JMeter tool and open the daytrader.jmx file in the
<PTThome>/workspace/Scripts/sample folder.
Change the hostname and port number (default is 80) to the host name and port

11. number of your proxy server.
Figure 8: Open the daytrader.jmx and modify the host name and port
Figure 9: Start the stress
Double click the host in the “Hosts” view of PTT workbench to connect to the cell
and start monitoring. It is recommended monitoring at least 10 minutes to make sure

12. no error will occur during normal stress testing.
Figure 10: Monitor your cell
Note: The dashboard will monitor all the key status of servers (include Application
Servers and Proxy Servers) in the latest monitoring interval (monitoring interval is
multiple of the data collection interval which was set in the preference panel)
The color of performance data will turn red if some abnormal events were detected in
current monitoring interval, it only represent the current status. There will also be a
warning mark in the second column as long as any error ever occurs and it will leave
unless you clear it. So the warning marks can represent history abnormal events.
Figure 10: Errors in the dashboard

13. Scenario1: “Synchronization Blocking” Problem
Application Changes to Simulate “Synchronization Blocking”
Situation
In this scenario, the application will continuously call static synchronized method, in
which the thread will sleep several seconds.
Sample code changes in the TradeAppServlet
//For Synchronization Blocking
if(TradeConfig.isThreadHang()){
syncMethod();
}
public static synchronized void syncMethod(){
try {
Thread.sleep(TradeConfig.getHangTime());
} catch (InterruptedException e) {
e.printStackTrace();
}
}
Trigger the “Synchronization Blocking” problem
Important: make sure that the stress is running and the PTT is monitoring before you
trigger the “Synchronization Blocking” problem
Click the “Server” tab of the home page (http://hostname:port/daytrader). The error
reproduction options will be shown (see figure11). In order to trigger the
synchronization blocking problems, select the “Synchronization Blocking” checkbox
in the “Server:” tab and update the runtime configuration. If you find that the hang
time is not long enough to trigger the thread hung problem, you can increase this
value accordingly.
.

14. Figure11. Error reproduction options
Monitor the “Synchronization Blocking” problem
When a synchronization blocking problem occurs, you will see the red data and
warning marks in the dashboard as below.
Figure12. Dashboard monitoring

15. Double click the server panel to switch to the monitoring page. In the servlet response
time section, you can see that the average response time keeps increasing.
Figure13. Server chart monitoring
Click the point to switch to the data page to check the servlet that is blocking.
Figure14. Check the detailed data
Besides, some servlet alerts will appear in the alert section.
Figure15. The alert panel
Click the alert point to switch to the data page, which show that the time limit of
servlet average response time is exceeded.

16. Figure 16: Alert monitoring data
Determine the “Synchronization Blocking” problem
 Generate a thread dump
In order to determine the “Synchronization Blocking” problem, we need to generate a
“Thread Dump” of the sick server from the content menu of the “Topology” view.

17. Figure 17: Take a thread dump of the sick server
 Launch the ISA 5.0 Tech Preview
Perform the following steps to launch ISA 5.0 Tech Preview.
1) Unzip the zip file to your file system.
2) Start WAS CE server with ISA5/start_isa.bat
3) Access ISA 5.0 with URL: http://localhost:8080/isa5/

18. Figure 18: GUI of ISA 5.0 Tech Preview
 Use ISA V5 to analyze the “Synchronization Blocking” problem
Add a new test case for the “Synchronization Blocking” problem.
Figure 19: Add new case

19. Figure 20: Input case info
Add the thread dump files generated to the test case. By default, they are generated in
the profile root folder of the sick server.
Figure 21: Add thread dump to the case
Next, analyze the thread dump file using the “Thread and Monitor Dump Analyzer”
tool of ISA v5 and view the analysis results.

20. Figure 22: Analyze the thread dump
Figure 23: Analysis finished
Figure 24: View the analysis results
You need to check the detail info of monitors and threads to find the root cause.

21. Figure 21: Analysis results summary
Figure 22: “Monitor Dump” analysis summary
Figure 23: “Monitor Dump” analysis detailed info
From the detailed page we can find the blocked threads and the status of lock owner,
so we can improve our application accordingly.

22. Scenario2: “Dead Lock” problem
Application Changes to Simulate “Dead Lock” Situation
In this scenario, a dead lock problem will be triggered and all the other threads will be
hung for ever, as they can not get the synchronization lock which is owned by the
deadlock threads.
Below is the code changes in the TradeAppServlet
//For Dead Lock
if(TradeConfig.isDeadLock()){
deadLockMethod();
}
public void deadLockMethod(){
try {
synchronized (lock1) { // lock1 is defined in the "Methods
and Static Variables" tab
Thread.sleep(5000);
synchronized (lock2) {
}
}
synchronized (lock2) { // lock2 is defined in the "Methods
and Static Variables" tab
Thread.sleep(5000);
synchronized (lock1) {
}
}
} catch (Exception e) {
e.printStackTrace();
}
}
Trigger the “Dead Lock” problem
Important: make sure that the stress is running and the PTT is monitoring before you
trigger the “Dead Lock” problem

23. In the “Server” tab of Daytrader home page, select the “Deadlock” checkbox and
click the “Update Configuration” button. A deadlock will be triggered. It will not only
block the deadlock lock threads, but also block other threads which have to wait for
the lock owned by the deadlock threads.
Figure24: Error reproduction options
Monitoring the “Dead Lock” problem
When a dead lock problem occurs, you will see the red data and warning marks in the
dashboard.

24. Figure25: Dashboard monitoring
Double click the server panel to switch to the monitoring page, and you can see that
some servlet alerts are shown in the alert section.
Figure26: Server monitoring
Click the alert point in the chart to check the alert content in the detailed data page.

25. Figure27: Alert detailed content
Determine the “Dead Lock” problem
 Generate a thread dump
In order to determine the “Dead Lock” problem, we need to generate a “Thread
Dump” of the sick server from the content menu of the “Topology” view.

26. Figure 28: Take a thread dump of the sick server
 Use ISA V5 to analyze the “Dead Lock” problem
Add a new test case for the “Dead Lock” problem.

27. Figure 29: Add new case
Figure 30: Add thread dump to the case

28. Figure 31: Analyze the thread dump
Figure 32: View the analysis results
Figure 33: “Monitor Dump” analysis summary
Figure 34: “Monitor Dump” analysis summary

29. Figure 35: “Monitor Dump” analysis detailed info
From the detailed page we can find the deadlock threads and their status, so we can
fix the problem accordingly. A best practice on this kind of problem is getting the
locks in the same order. So here is the improved code:
public void deadLockMethod(){
try {
synchronized (lock1) { // lock1 is defined in the "Methods
and Static Variables" tab
Thread.sleep(5000);
synchronized (lock2) { // lock2 is defined in the
"Methods and Static Variables" tab
}
}
synchronized (lock1) {
Thread.sleep(5000);
synchronized (lock2) {
}
}
} catch (Exception e) {
e.printStackTrace();
}
}

30. Scenario3: “High CPU Usage” problem
Application Changes to Simulate “High CPU Usage” Situation
In this scenario, a CPUConsumer thread will be started which will invoke an empty
loop method. You can increase the thread number if the CPU usage is not high
enough.
Sample code changes in the TradeAppServlet
//For High CPU Usage
if(TradeConfig.isHighCpuUsage()){
if(cpuThreadCount<TradeConfig.getThreadNum()){
try {
cpuThreadCount++;
CPUConsumer cc = new CPUConsumer();
cc.start();
} catch (Exception e) {
e.printStackTrace();
}
}
}
class CPUConsumer extends Thread{
public void run() {
try{
while(TradeConfig.isHighCpuUsage()){
}
cpuThreadCount--;
}catch(Exception e){
e.printStackTrace();
}
}
}
Trigger the “High CPU Usage” problem
In the “Server” tab of Daytrader home page, select the “High CPU Usage” checkbox
and click the “Update Configuration” button. A loop thread will be started to consume

31. the CPU resources. If you find that the CPU usage is not high enough to trigger the
“HighCPU” health policy, you can increase the thread number accordingly
Figure36: Error reproduction options
Monitoring the “High CPU Usage” problem
When a high CPU usage problem occurs, you will see the red data and warning marks
in the dashboard.

32. Figure37: Dashboard monitoring
Double click the server panel to switch to the monitoring page, and you can see that
the CPU usage is very high.
Figure38: Server CPU monitoring
Some runtime alerts are shown in the alert section.
Figure39: Alert panel
Click the alert point to check the alert content in the detailed data page

33. Figure39: Alert detailed content
Determine the “High CPU Usage” problem
In order to determine the High CPU Usage problem, we need to collect the CPU
usage info from the command line. But the command differs from OS to OS; here we
will take the Linux for example. Change the node name and server name of the sick
server and run the shell file with the following content and content.
node="node1"
server="TradeDC_node1"
echo Start get cpuInof on `date` >> cpuInfo.txt
echo
===================================================================
================================= >> cpuInfo.txt
pid=`ps -ef |grep java | grep com.ibm.ws.runtime.WsServer | grep -w "$node $server" |
awk '{print $2}'`
echo "$pid" >> cpuInfo.txt
if [ -n "$pid" ]
then

34. top -b -n1 -H -p $pid >> cpuInfo.txt
kill -3 $pid
fi
echo Finish custom action on `date` >> cpuInfo.txt
Open the cpuInfo.txt file generated by the “collectCPUInfo” custom action. You will
find the system CPU info as below:
Start custom action on Mon Feb 20 23:20:01 CST 2012
==================================================================================================
32094
top - 18:20:20 up 6 days, 8:25, 4 users, load average: 1.48, 1.38, 1.31
Tasks: 180 total, 1 running, 179 sleeping, 0 stopped, 0 zombie
Cpu(s): 6.6%us, 0.5%sy, 0.0%ni, 92.4%id, 0.1%wa, 0.0%hi, 0.4%si, 0.0%st
Mem: 4152360k total, 4029544k used, 122816k free, 162536k buffers
Swap: 2064376k total, 0k used, 2064376k free, 2350792k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
623 root 20 0 915m 494m 58m R 82.8 12.2 123:00.06 java
32255 root 20 0 915m 494m 58m S 2.0 12.2 0:14.72 java
32094 root 20 0 915m 494m 58m S 0.0 12.2 0:00.00 java
32096 root 20 0 915m 494m 58m S 0.0 12.2 0:18.93 java
32097 root 20 0 915m 494m 58m S 0.0 12.2 0:02.77 java
32098 root 20 0 915m 494m 58m S 0.0 12.2 0:26.89 java
32101 root 20 0 915m 494m 58m S 0.0 12.2 0:24.46 java
32102 root 20 0 915m 494m 58m S 0.0 12.2 0:00.00 java
32103 root 20 0 915m 494m 58m S 0.0 12.2 0:00.00 java
32104 root 20 0 915m 494m 58m S 0.0 12.2 0:00.00 java
32106 root 20 0 915m 494m 58m S 0.0 12.2 0:00.06 java
32107 root 20 0 915m 494m 58m S 0.0 12.2 0:00.06 java
32109 root 20 0 915m 494m 58m S 0.0 12.2 0:00.48 java
32115 root 20 0 915m 494m 58m S 0.0 12.2 0:00.00 java
32116 root 20 0 915m 494m 58m S 0.0 12.2 0:00.38 java
32118 root 20 0 915m 494m 58m S 0.0 12.2 0:00.02 java
32121 root 20 0 915m 494m 58m S 0.0 12.2 0:02.24 java
List 1: Content of cpuInfo.txt
From these info, we can find that the thread “623” consume 82.8% of CPU resource,
which is the root cause of high CPU usage problem.
As the thread dump will use “Hexadecimal” as the thread id, we need to convert the
thread pid from 623 (Decimal) to 0x26F (Hexadecimal) firstly, and then search it in
the thread dump file, you will find the root cause is the CPUConsumer.run method

35. in TradeAppServlet. Below is the stack trace:
3XMTHREADINFO "Thread-95" J9VMThread:0x0C0EFE00, j9thread_t:0x8D15AFA8, java/lang/Thread:0x9B2C2FC0, state:CW,
prio=5
3XMTHREADINFO1 (native thread ID:0x26F, native priority:0x5, native policy:UNKNOWN)
3XMTHREADINFO2 (native stack address range from:0x02B87000, to:0x02BC8000, size:0x41000)
3XMTHREADINFO3 Java callstack:
4XESTACKTRACE at
org/apache/geronimo/samples/daytrader/web/TradeAppServlet$CPUConsumer.run(TradeAppServlet.java:120)
3XMTHREADINFO3 Native callstack:
4XENATIVESTACK (0x00CA1752 [libj9prt24.so+0xb752])
4XENATIVESTACK (0x00CACF60 [libj9prt24.so+0x16f60])
4XENATIVESTACK (0x00CA17E5 [libj9prt24.so+0xb7e5])
4XENATIVESTACK (0x00CA1908 [libj9prt24.so+0xb908])
4XENATIVESTACK (0x00CA1584 [libj9prt24.so+0xb584])
4XENATIVESTACK (0x00CACF60 [libj9prt24.so+0x16f60])
4XENATIVESTACK (0x00CA15F8 [libj9prt24.so+0xb5f8])
4XENATIVESTACK (0x00C9D500 [libj9prt24.so+0x7500])
4XENATIVESTACK (0x0099740C)
List 2: Content of thread dump
From the content of cpuInfo.txt and the thread dumps, we can find the threads
consuming lots of CPU resources and their stack trace, so we will be able to fix the
problem accordingly.
Scenario4: “Connection Leak” problem
Application Changes to Simulate “Connection Leak” Situation
In this scenario, some requests will invoke a ConnectionConsumer thread in which it
will get a connection from the connection pool and never close it. You can adjust the
“Leak Frequency” to control the leak rate.
Code changes in the TradeAppServlet
//For Connection Consuming
if(TradeConfig.isConnectionLeak()){
if(triggerByFrequency(TradeConfig.getLeakFrequency())){
ConnectionConsumer cc = new ConnectionConsumer();
cc.start();
}

36. }
class ConnectionConsumer extends Thread{
public void run() {
try{
javax.naming.InitialContext ctx = new
javax.naming.InitialContext();
javax.sql.DataSource ds = (javax.sql.DataSource)
ctx.lookup("jdbc/TradeDataSource");
java.sql.Connection con = ds.getConnection();
}catch(Exception e){
e.printStackTrace();
}
}
}
Trigger the “Connection Leak” problem
In the “Server” tab of Daytrader home page, select the “Jdbc Connection Leak”
checkbox and click the “Update Configuration” button. It will leak the connection of
TradeDataSource at the specified frequency, and you can also modify the “Leak
Frequency” accordingly.

37. Figure40: Error reproduction options
Monitoring the “Connection Leak” problem
When a connection leak problem occurs, you will see the red data and warning marks
in the dashboard.
Figure41: Monitoring the dashboard
Double click the server panel to switch to the monitoring page, and you can see
several connection pool and servlet alerts in the alert section.
Figure42: Alert Panel
Go to the detailed data page to check the alert content.

38. Figure43: Detailed alert content
Go to connection pool detailed page to check the c
Figure44: Detailed connection pool data
Some other problems will be triggered after a while.

39. Figure45: Monitoring the dashboard
Determine the “Connection Leak” problem
In order to get the jdbc connection allocation stack trace, we need to enable the
“ConnLeakLogic” trace. In the “topology” view, right click the “sick” server and
select the “Enable Trace…” from the context menu.
In the popup dialog box, select the “ConnLeakLogic” trace and click “Ok”, button to
enable the trace.

40. Figure46: Enable the “ConnLeakLogic” trace
After several minutes, right click the “sick” server again and select the “Show
connection pool contents” from the context menu.
Figure47: Show connection pool contents
In the connection pool content window, select the “TradeDataSource”, you will be
able to check all the connections status, including the allocation time, used time, used
thread number and allocation stack trace.
Note: If you did not enable the “ConnLeakLogic” trace, the allocation stack trace will
not be able to shown here. Besides, we only be able to get the alloction stack trace
info for connections that allocated after we enable the “ConnLeakLogic” trace, other
connections’ allocation stack trace is not recorded.

41. Figure48: Connection pool contents
From the connection pool content info, we will be able to see that lots of connections
are used for more then 200 seconds by some threads, but all of them can not be found
in the thread dump, which means that the threads are completed but the connections
are not released, so it is a connection leak problem. Besides, nearly all of the leaked
connections were allocated by the following stack trace, so we can fix the problem in
our code accordingly.

42. Scenario5: “Memory Leak” problem
Application Changes to Simulate “Memory Leak” Situation
In this scenario, some requests will put some byte[] to the static ArrayList which will
never be released. You can adjust the “Leak Size” to control the leak rate.
Code changes in the TradeAppServlet
//For Memory Leak
public static ArrayList leakContainer = new ArrayList();
if(TradeConfig.isMemeoryLeak()){
leakContainer.add(new byte[TradeConfig.getLeakSize()]);
}
Trigger the “Memory Leak” problem
In the “Server” tab of Daytrader home page, select the “Memory Leak” checkbox and
click the “Update Configuration” button. It will leak the heap memory at the specified
rate.

43. Figure49: Error reproduction options
Monitoring the “Memory Leak” problem
When a connection leak problem occurs, you will see the red data and warning marks
in the dashboard.

44. Figure50: Monitoring the dashboard
Double click the server panel to switch to the monitoring page, and you can see
runtime alerts in the alert section.
Figure51: Alert Panel
Click the alert point to check the alert content in the detailed data page.
Figure52: Detailed Alert Contents
Determine the “Memory Leak” problem
In order to determine the memory leak problem, we need to generate a heap dump. In
the “topology” view, right click the “sick” server and select the “Generate heap
dump” from the context menu. The heap dump file will be generated in the profile
home by default.

45. Figure53: Generate a heap dump
Then add the heap dump into the "Case" in the ISA v5.
Figure54: Add heapdump file to you case
In "Tools" panel, Launch "Memory Analyzer" for JVM heap dump.

46. Figure55: Analyze the heap dump
Click "Launch", then Click "Browse" to select a dump file. Submit the task.
Figure56: Start to analyze
The task is running as a background task. A pop-up window appears to indicate the
output directory:
Figure57: Confirm dialog

47. The execution history and status are shown in the bottom of "memory analyzer'" page.
Figure58: Status panel
Memory analyzer takes several minutes to analyze the dump file. Please be patient
when the task is running background.
Check the output directory for the heapdump analysis and open the report file
"heapdump.*_analyzer.html".
Figure59: Check the analysis results
Select the "Leak suspects" reports for details. In this sample, "problem suspect 1"
consumed 237.4MB memory.

48. Figure60: Leak suspects
Expand the detail analysis for problem suspect 1, you will find the memory issue is
introduced by "ArrayList" objects in TradeAppServlet, which is the code changes we
did in section Application Changes to Sils -lmulate "Memory Leak" Situation.

49. Figure61: Detailed info of leak suspects
For more information about Memory Analyzer, visit http://www.ibm.com/developerworks/
java/jdk/tools/index.html
Other optional functions you can go through
Tuning the performance related parameters
There are 2 ways to change the settings of application server in the PTT. The first one
is via the “Tuning Parameters” view.

50. Figure62: Tune the perfomance parameters
The second way is executing tuning scripts in the scripts view. Some useful tuning
scripts have been added to the toolkit and you can make some modification base on
them.
Figure63: Run wsadmin scripts
Adjust rules from rule editor if needed

51. The WebSphere Application Server Performance Tuning Toolkit (PTT) allows users to define their
own rules according to the indicator of their system. For example, set a proper threshold for
response time of each servlet. All the customized rules for detecting performance decline and
locating the bottleneck are write in the rule file.
The existing rules:
· If the used heap reaches the 85% of maximum heap size, it will generate a runtime alert
· If the used thread pool reaches 90% of the max pool size, it will generate a threadpool alert
· If the average CPU usage reaches 90%, it will generate a runtime alert
· If some servlet errors occur, it will generate a servlet alert
· If some JDBC connection timeout occur, it will generate a connection alert
· If there are more then 1000 prepared statement discarded, it will generate a connection alert
· If there are some thread waiting for connection, it will generate a connection alert
· If some JCA connection error occur, it will generate a JCA alert
· If there is no room for new session, it will generate a session alert
· If some hung threads are declared, it will generate a threadpool alert
· If some transactions rolled back, it will generate a transaction alert
· If some transactions timeout, it will generate a transaction alert
· If some proxy request failed, it will generate a proxy alert
· If a servlet performance decline detect it will generate a servlet alter
· If a jdbc performance problem detect, it will generate a connection pool alert
All these rules are common rules, and you may need to modify them according to the
characters of your own system. For example, we can change the threshold of servlet
response time according to the performance data we collected to make the
performance problem detection rule more precise.

52. Figure64: Modify the rules
Generate report
You can also generate a report to get a clear view of your system.
Figure65: Generate a report
More Functions will be added later ……