MustGather: 100% CPU Usage on Solaris platforms
 Technote (FAQ)
 
Problem
MustGather for problems with the WebSphere® Application Server or other Java™ Virtual Machine (JVM™) experiencing 100% or unusually high CPU usage on Solaris™ platforms. Gathering this information before calling IBM support helps familiarize you with the troubleshooting process and saves you time.
 
Solution
The following document outlines what is needed to troubleshoot a JVM that causes the CPU utilization to reach 100%, or an unusually high percentage, on a Solaris system.


If you have already contacted support, continue on to the component-specific MustGather information. Otherwise, click: MustGather: Read first for all WebSphere Application Server products.


100% CPU Usage specific MustGather information
Follow these instructions to set up the failing System for capturing required information:

  1. Enable verbosegc for the failing application server. This is necessary for debugging High CPU issues and does not have significant impact on performance.

  2. Clear all application server log files before starting test. You may have to stop the Application Server to delete the files and then start them.

  3. After starting the Application Server, run the following netstat and vmstat commands to capture typical system CPU and connection activity:

    % netstat -an > netstat_normal.out
    % vmstat 5 12 > vmstat_normal.out



Collect the following information when the application server process is experiencing the problem:
  1. Execute the following script (see attached) where PID is the process ID of the Java process causing high CPU.

    ./wascpu.sh  PID

  2. Collect the following files:

    • For WebSphere V6.0:
      • All files in the following directory:

        install_root/profiles/profile_name/logs/server_name

      • A copy of server.xml located in the following directory:

        install_root/profiles/profile_name/config/cells/
        cell_name
        /nodes/node_name/servers/server_name

    • For WebSphere V5.0 and V5.1:
      • All files in the following directory:

        install_root\logs\server_name

      • A copy of server.xml located at the following directory:

        install_root\config\cells\cell_name\nodes\
        node_name
        \servers\server_name

    • For WebSphere V3.5, and V4.0:
      • All files in the following directory:

        install_root\logs

      • A copy of XMLExport for the server configuration.

    • If the Web server is remote, send the trace.log.Web_server_name.date (V3.5), native.log (V4.0), or http_plugin.log (V5.0 and V6.0) file from the Web server system.

    • All of the *.out files created by the wascpu.sh script.

    • netstat_normal.out and vmstat_normal.out (if possible).

  3. Follow instructions to Submitting Diagnostic Information to IBM Technical Support for Problem Determination.

If the thread dumps are not getting generated for kill -3, you may want to try using the alternate threading library, see Changing to the alternate threading library on Solaris 8.

For a listing of all technotes, downloads, and educational materials specific to the 100% CPU Usage component, search the WebSphere Application Server support site.
 
Related information
Enabling verbosegc in WebSphere Application Server
Determining which Java thread is consuming CPU
lsof download link
Steps to Getting Support
MustGather: Read first for all WebSphere
Troubleshooting Guide
Changing to the alternate threading library
 
wascpu.sh
 
 


Document Information


Product categories: Software > Application Servers > Distributed Application & Web Servers > WebSphere Application Server > 100% CPU Usage
Operating system(s): Solaris
Software version: 6.0
Software edition:
Reference #: 1115625
IBM Group: Software Group
Modified date: Apr 5, 2006