|
| Problem | MustGather for native leak problems with the WebSphere® Application Server Out of Memory component on Linux®. Gathering this information before calling IBM® support will help familiarize you with the troubleshooting process and save you time. | | | | Solution | The following steps outline how to troubleshoot java.lang.OutOfMemoryError errors on a Linux platform caused by the inability of the Java™ virtual machine (JVM™) to allocate native memory. This suspicion is based upon your analysis of the documentation collected from the technote: MustGather: Out of Memory Errors on Linux, Part 1.
If you already contacted support, continue to the component-specific MustGather information. Otherwise, click: MustGather: Read first for all WebSphere Application Server Products.
Out of Memory (native leak) specific MustGather information:
- Enable tracing of native memory allocation usingIBM_MALLOCTRACE and MALLOC_TRACE to create mtrace.log:
- For WebSphere Application Server V6.0 release:
- Enable these environment variables through the administrative console by selecting the following:
Servers > Application Servers >server_name> Java and Process Management > Process Definition > Environment Entries > New
- Add the following Name and Value pairs:
Name: IBM_MALLOCTRACE Value:1
Name: MALLOC_TRACE Value:memory_log_path
Where memory_log_path is install_root/profiles/profile_name/logs/server_name/mtrace.log
- For WebSphere Application Server V5.0 and V5.1 releases:
- Enable these environment variables through the administrative console by selecting the following:
Servers > Application Servers >server_name> Process Definition > Environment Entries > New
- Add the following Name and Value pairs:
Name: IBM_MALLOCTRACE Value:1
Name: MALLOC_TRACE Value:memory_log_path
Where memory_log_path is install_root/logs/server_name/mtrace.log
- For WebSphere Application Server V3.5 and V4.0 releases:
Add the following lines to the top of the startupServer.sh file in the install_root/bin directory: export IBM_MALLOCTRACE=1
export MALLOC_TRACE=memory_log_path
Where memory_log_path is install_root/logs/mtrace.log
- Follow instructions for Enabling verbosegc in WebSphere Application Server.
- Delete all WebSphere Application Server and FFDC log files. Back up them first, if needed.
- After starting WebSphere Application Server, run the Linux_memory_leak.sh shell script (attached) while trying to reproduce the problem. Ensure that you update the following lines of the script before running it:
- log=file
- pid=PID_of_Java_process_that_leaks_memory
- Change the amount of time the script sleeps, "sleep 600", based on how long it takes to reproduce the problem.
For example, if the problem can be reproduced in 1 hour, update the sleep interval to 300 so that the documentation is collected every 5 minutes instead of 10. For a problem that takes 2 days to reproduce, 1200 is a good sleep interval.
- Wait for problem to re-occur.
- Stop the Application Server and the script.
- Collect the following files:
- For WebSphere Application Server V6.0 release:
- The systemErr.log, systemOut.log, native_stderr.log and native_stout.log files from install_root/profiles/profile_name/logs/server_namedirectory
- Everything in the install_root/profiles/profile_name/logs/ffdc directory
- Server.xml file from the directory:
install_root/profiles/profile_name/config/cells/cell_name/nodes/node_name/servers/server_name - All documents requested for all releases below
- For WebSphere Application Server V5.0 and V5.1 releases:
- The systemErr.log, systemOut.log, native_stderr.log, native_stout.log and startServer.log files from install_root/logs/server_namedirectory
- Everything in the install_root/logs/ffdc directory
- Server.xml file from the directory:
install_root/config/cells/cell_name/nodes/node_name/servers/server_name - All documents requested for all releases below
- For WebSphere Application Server V3.5 and V4.0 releases:
- The stderr, stdout and tracefile files from install_root/logs/
- Full XMLConfig export.
- All documents requested for all releases below
- For all releases
- mtrace.log
- The Linux_memory_leak.sh log file
- List of all other software and version information on the system. For example, any databases and WebSphere MQ®.
- Follow instructions to send diagnostic information to IBM support.
For a listing of all technotes, downloads, and educational materials specific to the Out of Memory component, search the WebSphere Application Server support site. | |
| | |  |
| |
|
Product categories: Software, Application Servers, Distributed Application & Web Servers, WebSphere Application Server, Out of Memory Operating system(s): Linux Software version: 3.5, 4.0, 5.0, 5.1, 6.0 Software edition: Advanced, Base, Express, Network Deployment, Single Server, Standard Reference #: 1138462 IBM Group: Software Group Modified date: 2004-12-15
(C) Copyright IBM Corporation 2000, 2004. All Rights Reserved.
|