The following steps outline how to
troubleshoot java.lang.OutOfMemoryError errors on a Linux platform caused by the inability of the
Java™ virtual machine (JVM™) to allocate native memory. This suspicion is
based upon your analysis of the documentation collected from the
technote: MustGather: Out of Memory Errors on Linux, Part
1.
If you already contacted support, continue to the
component-specific MustGather information. Otherwise, click: MustGather: Read first for all WebSphere Application Server
products.
Out of Memory (native leak) specific MustGather
information
- Enable tracing of native memory allocation
using IBM_MALLOCTRACE and
MALLOC_TRACE to create
mtrace.log:
- For WebSphere Application Server V6.0
release:
- Enable these environment variables through the
administrative console by selecting the following:
Servers > Application Servers
> server_name> Java and Process Management > Process Definition >
Environment Entries > New
- Add the following Name and Value pairs:
IBM_MALLOCTRACE |
1 |
MALLOC_TRACE |
memory_log_path |
|
 |
Where, memory_log_path is
install_root/profiles/
profile_name/logs/server_name/mtrace.log |
|
 |
- For WebSphere Application Server V5.0
and V5.1 releases:
- Enable these environment variables through the
administrative console by selecting the following:
Servers > Application Servers
> server_name> Process Definition > Environment Entries > New
- Add the following Name and Value pairs:
IBM_MALLOCTRACE |
1 |
MALLOC_TRACE |
memory_log_path |
|
 |
Where, memory_log_path is
install_root/logs/
server_name/mtrace.log |
|
 |
- For WebSphere Application Server V3.5
and V4.0 releases:
Add the following lines to the top of the startupServer.sh
file in the
install_root/bin directory:
export IBM_MALLOCTRACE=1
export MALLOC_TRACE=memory_log_path |
|
Where,
memory_log_path is
install_root/logs/mtrace.log |
|
- Follow instructions for Enabling verbosegc in WebSphere
Application Server.
- Delete all WebSphere Application Server and FFDC
log files. Back up them first, if needed.
- After starting WebSphere Application Server, run
the Linux_memory_leak.sh shell script
(attached) while trying to reproduce the problem. Ensure that you update
the following lines of the script before running it:
- log=file
-
pid=PID_of_Java_process_that_leaks_memory
- Change the amount of time the script sleeps, "sleep
600", based on how long it takes to reproduce the problem.
For example, if the problem can be reproduced in 1 hour, update the sleep
interval to 300 so that the documentation is collected every 5 minutes
instead of 10. For a problem that takes 2 days to reproduce, 1200 is a
good sleep interval.
- Wait for problem to re-occur.
- When the problem occurs issue the following to
produce a javacore*.txt file:
kill -3
AppServer_java_processID |
|
- Stop the Application Server and the script.
- Collect the following files:
- For WebSphere Application Server V6.0
release:
- The systemErr.log, systemOut.log, native_stderr.log
and native_stout.log files from
install_root/profiles/profile_name/logs/server_name directory
- Everything in the
install_root/profiles/profile_name/logs/ffdc
directory
- Server.xml file from the
directory:
install_root/profiles/profile_name/config/cells/
cell_name/nodes/node_name/servers/server_name
- All documents requested for all
releases below
- For WebSphere Application Server V5.0
and V5.1 releases:
- The systemErr.log, systemOut.log, native_stderr.log,
native_stout.log and startServer.log files from
install_root/logs/server_name directory
- Everything in the
install_root/logs/ffdc directory
- Server.xml file from the
directory:
install_root/config/cells/cell_name/nodes/
node_name/servers/server_name
- All documents requested for all
releases below
- For WebSphere Application Server V3.5
and V4.0 releases:
- The stderr, stdout and tracefile files from
install_root/logs/
- Full XMLConfig export.
- All documents requested for all
releases below
- For all
releases
- mtrace.log
- The
Linux_memory_leak.sh log file
- javacore*.txt
- List of all other software and
version information on the system. For example, any databases and
WebSphere MQ®.
- Follow instructions to send diagnostic information to IBM support.
For a listing of all technotes, downloads, and
educational materials specific to the Out of Memory component, search
the WebSphere Application Server support
site.
|