MustGather information for web server crashes (Windows)

System preparation

The Dr. Watson tool supplied with Windows must be configured to save information about program crashes. This setup step must be performed prior to the crash.

  1. Start DRWTSN32.EXE from the Command Prompt. The Dr. Watson configuration dialog will look similar to the following:

  2. Note that on Windows 2000 the "Crash Dump Type" option is not displayed. If your version of Dr. Watson has the capability to select Crash Dump Type, then select Full.
  3. Make a note of the paths of the log file and crash dump file or change the paths to point to an alternate location. After a crash has occurred, the DrWatson log file and user.dmp files will be retrieved from those locations.
  4. Enable the following options:
  5. We recommend that the following option be disabled so that IBM HTTP Server recovers as quickly as possible after a crash:
  6. Ensure that Number of Instructions is at least 10.
  7. Ensure that Number of Errors to Save is at least 10.
  8. Click "OK" to save these settings.
  9. From a command prompt, enter drwtsn32 -i to enable Dr. Watson as your default Windows debugger. This will make the necessary changes in the Windows registry.

Obtaining information after the crash occurs

These steps must be performed after the crash occurs.

  1. Crash information

    Save the Dr. Watson log and crash files. These are in the locations configured in the DRWTSN32.EXE program, as displayed above under System preparation. Example:

    C:\Documents and Settings\All Users\Documents\DrWatson>dir
     Volume in drive C is C_Drive
     Volume Serial Number is 98E7-6D3C
    
     Directory of C:\Documents and Settings\All Users\Documents\DrWatson
    
    08/05/2003  09:32a      <DIR>          .
    08/05/2003  09:32a      <DIR>          ..
    04/05/2005  03:48p           1,118,951 drwtsn32.log
    04/05/2005  03:49p          12,969,444 user.dmp
                   2 File(s)     14,088,395 bytes
                   2 Dir(s)  18,162,450,432 bytes free
    

    The files drwtsn32.log and user.dmp are the files to save.

  2. System and web server version and configuration information

    Use the System and web server information tool to gather this information. This will result in a directory of information called ServerConfig.timestamp, which should then be zipped.

  3. web server configuration file

    The default file is

    install_root\conf\httpd.conf
    
  4. web server error log

    The default file is

    install_root\logs\error.log
    

    The configuration file may have been changed to specify a different error log.

  5. WebSphere plug-in trace file

    The actual location is specified in plugin-cfg.xml and is generated by configuring LogLevel="Trace".

    Example:

    c:\WebSphere\AppServer\logs\http_plugin.log
    
  6. Windows system information

    Run Start/Programs/Accessories/System Tools/System Information, highlight the top level "System Information" selection, right mouse click, select "Save as Text File", enger "system_info" for the filename to save as, and then "Save" (this may take a few minutes):

Recap of information to send to IBM support:

  1. Dr. Watson log and crash files
  2. a .zip archive of the ServerConfig.timestamp directory
  3. web server configuration file
  4. web server error log
  5. WebSphere plug-in trace file
  6. Windows system information, as exported to file system_info.txt by the System Information program
Instructions to send diagnostic information to IBM support.

Known Windows drwtsn32.log crashes:

In order to identify a known crash from drwtsn32.log, first open the log file in a text editor, e.g. notepad.exe. Move to the bottom of the file, then select Edit/Find, check Match-case, select Direction-Up, and enter the Find-what text "FAULT" (do not include the quotes).

Known crashes can be identified by looking for the text indicated below. It will be either close to the FAULT pointer or in the Function Name of the Stack Back Trace just below the FAULT pointer.

Search text Problem Resolution
ap_die IHS 2.0.42.2: Fixed in APAR PQ85834
IHS 2.0.47: Fixed in IHS 2.0.47.1
gsk_attribute_get_data Fixed in mod_ibm_ssl (not GSKit) at current recommended service levels
In theory could happen on other platforms but so far the crash has only been seen on Windows

Other Known Windows crashes:

IHS fails to start when LDAPCodepageDir is configured

If there is a copy of the IBM LDAP Client or Server library already installed on the same machine as IHS it may have configured the TISDIR variable in the Windows environment. This conflicts with the IHS LDAPCodepageDir directive and may cause IHS to crash without any error or Windows event being logged. If if it is an older version which is no longer required, it is recommended that the IBM LDAP libraries be uninstalled in order to delete the TISDIR environment variable. If that is not possible then the LDAPCodePageDir directive must be removed and the default IHS LDAP codepage directory used.

IHS crashes on startup, or on restart when MaxRequestsPerChild is reached

Certain third party software may add a Winsock Service Provider layer which sits between IHS and the TCP/IP network. If this software does not implement some of the Microsoft advanced Winsock functions correctly, such as AcceptEx, it may cause IHS to crash on startup or restart. A possible fix is to add the "Win32DisableAcceptEx" directive to the IHS configuration file. Setting "MaxRequestsPerChild 0" (the recommended default) may also eliminate the crashes. This crash may be accompanied by the following message in the error log:

[warn] (OS 10038)An operation was attempted on something that is not a socket.  
: setsockopt(SO_UPDATE_ACCEPT_CONTEXT) failed.