|
Problem(Abstract) |
This MustGather explains how to force a user.dmp file.
IBM® WebSphere® Application Server Administrators should follow these
directions when experiencing a hang or degradation in performance and
javacores are not being produced through DrAdmin or wsadmin on the
Microsoft® Windows® operating system. Gathering this information before
calling IBM support will help familiarize you with the troubleshooting
process and save you time. |
|
|
|
Resolving the
problem |
If you have already contacted support, continue on to the hang or
performance degradation MustGather information. Otherwise, click: MustGather:
Read first for all WebSphere Application Server products.
Steps to setup a Microsoft Windows environment to generate Dr. Watson
logs and Userdump files
- On the Start menu, click Run.
- Enter drwtsn32, and then click OK.
This will display the DrWatson window.
- Type the location to where you want the Dr Watson log to be
stored in the field provided for Log File Path.
- Type the location and the file name where you want the
user.dmp file to be stored in the field provided for Crash
Dump.
- Set Crash Dump Type radio button to Full.
- Make sure that the following options are checked in DrWatson screen:
- Dump Symbol Table
- Dump All Thread Contexts
- Visual Notification
- Create Crash Dump file
- For Windows 2003, Crash Dump Type option needs to
be set to Windows NT® 4 compatible Full
- Click OK.
- Click Run on the Start menu.
- Enter drwtsn32 -i and click OK. This will enable
DrWatson to become the default debugger, and it will generate the DrWatson
log and Userdump when an application crashes.
Generating the user.dmp files during a hang situation
The following steps can be used to generate a series of user.dmp
files, without interrupting the execution of the java process.
The Userdump.exe tool can be used to generate a
user.dmp file of a process that stops responding (hangs). In most
cases, this tool will not kill the Java™ Virtual Machine (JVM) and will
allow for multiple user.dmp files to be taken during the time of
the problem.
- Download the Userdump
Tool. Extract the files to a safe directory of your choosing. There is
no need to install or run setup, the Userdump.exe tool can be used
directly from the archive.
- Gather netstat data. Enter the following from a command
prompt on the WebSphere Application Server machine and any remote Web
server machines:
netstat -an >
netstat_before.out |
|
- Create the first of three Userdump files for the hanging process:
- Run the Setup.exe program that is included with the
Userdump tool from the userdump folder.
When the Application Server process hangs, from a command prompt, enter
the following command:
userdump pid |
|
Where pid is the process ID of the hanging
java.exe. There are multiple ways to get the PID for the failing
application server. The following are a couple of ways:
- You can look in to the SystemOut.log file for the failing
application server and get the PID from the latest server startup log
entries.
- You can find a file named server_name.pid in
the logs folder for the failing server, that has PID for the
server.
|
|
- This produces java.dmp file in the directory from where the
userdump command was executed.
- Rename the java.dmp file to java1.dmp.
- Wait 2 minutes.
- Create the second of three userdump files for the hanging processes,
renaming java.dmp to java2.dmp.
- Create the third of three userdump files for the hanging processes,
renaming java.dmp to java3.dmp.
Note: If the preceding steps do not appear to work, see the section
later in this document named Alternative method for using drwtsn32 to
force a user.dmp However, the alternative method kills the
java process. |
|
- Run the following command against each of the java.dmp files:
For WebSphere Application Server V6.1:
jextract [DUMP_PATH] javadump.sdff
For WebSphere Application Server Versions V5.0, V5.1, and V6.0:
jextract -o javadump.sdff [DUMP_PATH]
Where:
jextract - is found in the install_root/java/jre/bin directory.
javadump.sdff - is the output file.
DUMP_PATH - is the path and file name for the java.dmp.
- Gather netstat data. Enter the following from a command
prompt on the WebSphere Application Server machine and any remote Web
server machines:
netstat -an >
netstat_after.out |
|
- Collect the following data:
- For WebSphere Application Server V6.0:
- The server.xml file located in the following
directory:
profile_root\config\cells\cell_name\nodes\
node_name\servers\server_name |
|
- plugin-cfg.xml and http_plugin.log
- Everything in the following directory:
profile_root\logs\server_name |
|
- Everything in the following directory:
- For WebSphere Application Server V5.1 and 5.0:
- The server.xml file located in the following
directory:
install_root\config\cells\nodes\node_name\
servers\server_name |
|
- plugin-cfg.xml and http_plugin.log
- Everything in the following directory:
install_root\logs\server_name |
|
- Everything in the following directory:
- For all releases:
- All java.dmp files
- All javadump.sdff files
- netstat*.out files
- Include the Application Server systemErr.log,
systemOut.log, native_stderr.log, and
native_stdout.log, if they are located in a different directory
- Web server access and error logs
- If the Web server is remote, send the appropriate file
from the Web server system including Web server configuration files and
Web server logs. For example, for IBM HTTP Server and Apache Web server,
provide the httpd.conf, access and error logs.
For the SunOne 6 Web server, provide magnus.conf,
obj.conf, access and error logs.
- Follow instructions to send
diagnostic information to IBM support.
Alternative method for using drwtsn32 to force a user.dmp
This is not the preferred method, since the Java process will be killed,
but can be used in situations where no other option works.
Drwtsn32 can be used to force a user.dmp file. This should
already be loaded on the system. This will kill the JVM and it will need
to be restarted.
- Enter the following from a command prompt on the WebSphere Application
Server machine and any remote Web server machines:
netstat -an >
netstat_before.out |
|
- Determine the pid of the WebSphere Application Server process
(review step 3a. from the preceding section, Generating the user.dmp
files during a hang situation).
- Issue the following command from a Windows command prompt (where
pid is the pid of the JVM that will not produce a javacore):
drwtsn32 -p
pid |
|
A user.dmp file should be created in the
following directory, or whatever location you previously set (See step 4
from the section, Steps to setup a Microsoft Windows environment to
generate Dr. Watson logs and Userdump files): |
|
operating_system_root\Documents and Settings\
All Users\Application Data\Microsoft\Dr Watson |
|
- Send in the generated user.dmp file and all applicable files
and logs from step 5 in the preceding section.
- Follow instructions to send
diagnostic information to IBM support.
For a listing of all technotes, downloads, and educational materials
specific to a hang or performance degradation, search the WebSphere
Application Server support site. |
|
|
|
|
|
Cross Reference information |
Segment |
Product |
Component |
Platform |
Version |
Edition |
Application Servers |
WebSphere Application Server - Express |
Hangs/performance degradation |
Windows |
6.0, 5.1, 5.0 |
|
Application Servers |
Runtimes for Java Technology |
Java SDK |
|
|
|
|
|
|