If you have already contacted support, continue on to the
component-specific MustGather information. Otherwise, click: MustGather:
Read first for all WebSphere Application Server products.
100% CPU Usage specific MustGather information
This document lists what is needed to begin troubleshooting a Java™
Virtual Machine (JVM™) that pegs your CPU to 100% (or an unusually high
percentage) on a Windows system.
Windows NT/2000 includes a performance viewer (part of the Microsoft
management console) that can be used to determine which threads in a Java
process are consuming the CPU.
Follow these instructions to setup the failing System for capturing
required information:
- Follow the instructions to enable
verbosegc for the failing application server.
- Clear all application server log files before starting test. You may
have to stop the application server to delete the files.
Collect the following information when the problem is the
application server process:
- From the Start menu, select Programs > Administrative
Tools > Performance Monitor. (Alternatively, you might find
Start > Settings > Control Panel, double-click
Administrative Tools, and double-click Performance to open
the Performance Monitor tool.)
- Performance data can be captured to a log file. The log file can be
forwarded to a location for analysis and rerun through the performance
monitor. From the performance monitor, expand Performance logs and alerts,
right-click Counter Logs, New Log Settings.
- Type a log setting name, then click OK.
- Click Add to add counters.
- In the Performance Object drop down window, select Thread as
Object.
- Click Select counter from List (default), then select the
following counters:
Counter = % Processor Time
Counter = ID Process
Counter = ID Thread
- Click Select instances from list and select all Java threads as
shown in this example:
- Click Add to add new properties, then click Close.
- Logging starts immediately.The log file icon turns green.
- While the performance data is being logged, gather relevant
Performance Degradation documentation. You can get instructions for
gathering performance degradation, by selecting the appropriate link for
your WebSphere Version:
V4
V5
V6
This is a very important step. Support requires this
information to determine the cause of the CPU consumption.
- Once Performance Degradation documentation is collected, end the
Performance Monitor logging. From the Performance Monitor tool,
right-click Log Name in the counter window, then click Stop.
The log file to send to IBM is referenced in the Log File Name
column.
- Generate a user.dmp of the Java process. Issue the following command:
drwtsn32 -p <PID of Java process>
- Collect the following files
- The perfmon log
- The javacore*.txt files generated
- The user.dmp file generated from step 11. This is most
likely located in the C:\Documents and Settings\All
Users\Documents\DrWatson directory.
- Remaining information referenced in the Performance
Degradation documentation.
- For V6.0 release:
- All files in
<install_root>/profiles/<ProfileName>/logs/<ServerName>directory.
- A copy of server.xml located in
<install_root>/profiles/<ProfileName>/config/cells/<CellName>/nodes/<NodeName>/servers/<ServerName>
directory.
- For V5.0 and V5.1 releases:
- Include all of the files from the
<install_root>\logs\<ServerName> directory.
- A copy of server.xml located at
<install_root>\config\cells\<CellName>\nodes\<nodeName>\servers\<ServerName>
directory
- For V3.5 and V4.0 releases:
- Include all of the files from the
<install_root>\logs directory.
- A copy of XMLExport for the server configuration
- Follow instructions to send
diagnostic information to IBM support.
Analyzing the data
- To review the log, click the View Log File Data icon on the
tool bar in the System Monitor View, and open the performance log
file.
- Click the plus ( + ) button. In the Add To Chart window, select the
following items:
Object = Thread
Counter = % Processor Time
Counter = ID Process
Counter = ID Thread
And select all instances.
- Determine the thread ID that consumes the CPU
- Convert the thread ID (3480) to hexadecimal using the Windows
calculator
3480 in decimal = D98 in hex
- In the javacore.txt file, locate the thread that has a native
ID of 0xD98
For example:
"SoapConnectorThreadPool : 0" (TID:0x10A25A98,
sys_thread_t:0x24A31B30, state:R, native ID:0xD98)
prio=5
- Review the stack of that thread in each javacore to determine the
cause of the CPU usage.
For a listing of all technotes, downloads, and educational materials
specific to the 100% CPU Usage issue, search the WebSphere
Application Server support site.
|