Using IBM HTTP Server diagnostic capabilities with WebSphere

Problems occasionally encountered with WebSphere applications include

• unresponsive behavior
• slow response times

When WebSphere requests are routed through IBM HTTP Server using the WebSphere plug-in, the same symptoms may have different causes, such as:

• problems with authentication modules used within IBM HTTP Server
• network problems encountered by WebSphere plug-in when communicating with WebSphere
• other issues within IBM HTTP Server which are unrelated to WebSphere

Preparation

Before a problem occurs, enable these IBM HTTP Server features.

Enable mod_status and ExtendedStatus

mod_status provides a report of current web server activity. This includes the current set of requests which are being processed, the length of time that the request has been active, and the active plug-in module (if any) which is handling the request.

In order to see details about each request, the ExtendedStatus directive must be set to On in httpd.conf.

Add response time to the access log

When there are concerns about page display time, it is important to know which types of requests are responded to slowly. By recording the response time for every request in the access log, analysis of the access log can provide information about where the problem may lie.

The response time in seconds can be added to the end of every line of the access log by adding %T to the end of the format string used for logging. This format string is configured using the LogFormat directive in httpd.conf.

In the following example, a new format called responsetime has been created with the response time in seconds at the end of the line, and the access log uses that format.

LogFormat "%h %l %u %t \"%r\" %>s %b %T" responsetime
...
CustomLog logs/access_log responsetime

Note:Any programs used in your environment to analyze the access log may be impacted by adding this additional field. If there are such programs in use, investigate their requirements before implementing the change.

The LogFormat and CustomLog directives are described in the mod_log_config documentation.

Diagnosing unresponsive behavior

In order to collect documentation from the appropriate component, it is important to determine if the unresponsive behavior is caused by an issue within IBM HTTP Server or within WebSphere. The easiest way to check this is to connect directly to the WebSphere port with the browser and run the application in that manner.

If the application doesn't respond normally when connecting directly to WebSphere, then the problem is not expected to involve IBM HTTP Server or the WebSphere plug-in. Consult the WebSphere MustGather documents to collect the appropriate documentation.

If the application responds normally when connecting directly to WebSphere but the application does not respond when the application request is routed through IBM HTTP Server, then the problem is not expected to involve WebSphere.

The next step is to view the server status page, including the optional module display. The URL will be http://www.example.com/server-status/?showmodule. Substitute your own hostname for www.example.com.

The bottom portion of the report will look something like this:

Ignore entries with _ in the M column, since those IHS threads are idle. The SS column contains the number of seconds since the start of the request.

For requests with an unexpectedly long time in the SS column, check the value in the Module column. If the module is mod_was_ap20_http.c or mod_app_server_http.c, then the request is currently being processed by the WebSphere plug-in. Check the WebSphere plug-in log file for processing problems. If you were unable to determine if the application responds normally when connecting directly to WebSphere, perhaps due to application authentication requirements which are fulfilled by modules enabled in IBM HTTP Server, then the issue is most likely in WebSphere.

If the module which is processing the stalled request is a third-party module, such as one that handles authentication, consult the vendor of that module for instructions on diagnosing the problem.

Otherwise, follow IBM HTTP Server MustGather instructions for web server hangs. (Unix and Linux systems) (Windows systems)

Diagnosing slow response times

A common cause of slow response times for all types of requests is an insufficient number of IBM HTTP Server worker threads configured in httpd.conf. During peak loads, requests may be queued for some time until a worker thread becomes available to handle the request. This tuning is described in the performance tuning guide.

If there is not a problem with the number of configured worker threads, check the response times in the access log to see which requests are responded to slowly. Here are some possible conclusions:

• all requests are responded to slowly

  Check for network problems. Check for excessive load or paging on the web server machine.

• requests for static files such as images are responded to quickly but requests for dynamic application content are responded to slowly

  Investigate the generator of the dynamic application content (e.g., WebSphere application).

• requests for some application content are responded to slowly, but requests for other application content are responded to quickly

  Investigate the generator of the dynamic application content (e.g., WebSphere application).

• requests which must be authenticated are responded to slowly, but unauthenticated requests are responded to quickly

  Check with the vendor of the authentication module to determine the appropriate diagnosis steps. This may include checking LDAP or other servers, as well as checking the log written by the authentication module.