MustGather: Out of Memory errors on Windows, Part 2a - Native Leak
 Technote (FAQ)
 
Problem
MustGather for troubleshooting WebSphere® Application Server java.lang.OutOfMemoryError errors on the Windows® platform when you suspect a memory leak in the Native Heap. Base this action on your analysis of the documentation collected from the Technote MustGather: Out of Memory errors on Windows, Part 1.
 
Solution
This is the second step in gathering troubleshooting data when there is evidence of a memory leak. MustGather: Out of Memory Exceptions on Windows, Part 1 is a prerequisite, since it helps you decide if the procedures in this technote are necessary.

If you have already contacted WebSphere Technical Support, continue to the component-specific MustGather information. Otherwise, click the following link: MustGather: Read First for all WebSphere Application Server Products.


Out of Memory (native memory) specific MustGather information
Windows includes a performance viewer (part of the Microsoft® Management Console) that can be used to determine which threads in a Java™ process are consuming memory. Please start the performance viewer using the following steps before you start the Application Server that is experiencing the problem:
  1. From Start, select Programs > Administrative Tools > Performance.

    Alternatively, from Start, select Settings > Control Panel, then double-click on Administrative Tools. Double-click on Performance to open the Performance Monitor tool.

  2. Expand Performance Logs and Alerts.

  3. Right-click Counter Logs, then select New Log Settings and enter a name for the log in the New Log Settings dialog. The Log Settings dialog appears.

  4. Click Add Counters to add counters. The Add Counters dialog appears.

  5. In the Performance object drop down list, select Objects.

  6. Click Select counters from list, then select the following counters:


    Counter = mutexes
    Counter = semaphores
    Counter = threads

  7. Click Add to add new properties.

  8. In the Performance object drop down list, select Memory.

  9. Click Select counters from list, then select the following counters:


    Counter = available Mbytes
    Counter = page faults/sec
    Counter = page reads/sec
    Counter = page writes/sec


  10. Click Add to add new properties, then click Close. Logging starts immediately.


Once you receive a java.lang.OutOfMemory error, follow these steps to stop the Performance viewer and collect the necessary log data:
  1. To end logging, right-click Log Name in the counter window, then click Stop.

  2. To generate a user.dmp file of the java process, issue the following command:


    drwtsn32 -p <PID_of_Java_process>


    This will kill the java process. To determine the location of the generated user.dmp file click Start > Run > type drwtsn32. This will open the Dr. Watson for Windows application. The location of the user.dmp file will be indicated in the Crash Dump path.


  3. Collect the following data:
For WebSphere V6.0:
    • The server.xml file located in the following directory:

      install_root\profiles\profile_name\config\cells\
      cell_name
      \nodes\node_name\servers\server_name

    • Everything in the following directory:

      install_root\profiles\profile_name\logs\server_name

      Specifically, the following logs are needed:
      • systemErr
      • systemOut
      • native_stderr
      • native_stdout

    • Everything in the following directory:

      install_root\profiles\profile_name\logs\ffdc

    • All documents requested for all releases (see below).

  • For WebSphere V5.0 and V5.1:
    • The server.xml file located in the following directory:

      install_root\config\cells\nodes\node_name\servers\server_name

      Specifically, the following logs are needed:
      • systemErr
      • systemOut
      • native_stderr
      • native_stdout

    • Everything from the following directory:

      install_root\logs\server_name

    • Everything from the following directory:

      install_root\logs\ffdc

    • All documents requested for all releases (see below).

  • For WebSphere of V4.0 and V3.5:
    • A XMLConfig full export.

    • Everything from the following directory:

      install_root\logs

      Specifically, the following logs are needed:
      • stderr
      • stdout
      • tracefile

    • All documents requested for all releases (see below).

  • For all releases:
    • Perfmon logs: Logs created by the performance tool.
    • user.dmp file generated in step 2, above.
    • All javacore.txt files that were created when OutOfMemory conditions occurred. If javacore.txt files were created, you can find them in the install_root, install_root\bin or in the configured working directory
    • Include the Application Server systemErr, systemOut, native_stderr, and native_stdout logs, if they are located in a different directory

Follow instructions to send diagnostic information to IBM support.

For a listing of all technotes, downloads, and educational materials specific to the Out of Memory component, search the WebSphere Application Server support site.
 
Related information
Out of Memory Errors on Windows - Part 1
Submitting information to IBM support
Steps for getting support
MustGather: Readme first
Troubleshooting guide
 
 
 


Document Information


Product categories: Software > Application Servers > Distributed Application & Web Servers > WebSphere Application Server > Out of Memory
Operating system(s): Windows
Software version: 6.0
Software edition:
Reference #: 1153585
IBM Group: Software Group
Modified date: Apr 5, 2006