PQ98243: ABENDDC3 RC=000C0006 AND C9C20020 IN WEBSPHERE CONTROLLER REGION, AFTER MDB TIMEOUTS IN SERVANT REGION

 A fix is available

Obtain the fix for this APAR



APAR status
Closed as program error.

Error description
WebSphere application server was processing MDB workload, and
a MDB timeout occurred (abend EC3 reason 04130008) AND that
MDB was running under a transaction.  The application servant
region was in the process of being abended, and a new one
started by WLM.  While the CR was processing this one MDB
timeout for the servant region, another MDB request timed out
for the same servant region and that MDB was also running under
a transaction.  In this case, the quick succession of timeouts
of requests also running under transactions caused the
application controller region to abend with ABEND DC3 reason
000C0006 indicating another error occurred.  The "other error"
can be seen in the controller region job output as minor code
C9C20020 indicating an internal ras_assert was done.  The
ras_assert is surfaced in the controller region job output
in the trace as
Trace: 2004/11/18 16:10:05.917 01 t=AC6088 c=UNK key=S2
(0000000A)
  Description: Log Boss/390 Error
  from filename: ./bbottmcr.cpp
  at line: 3492
  error message: BBOO0021E Assertion failed:  ctl_p != NULL ,
  file ./bbottmcr.cpp, line 3492.
This was preceded by minor code c9c21948 from bbotrrm.cpp line
2122 on
the same thread t=AC6088 at 16:10:05.887.
C9C21948 indicates ATR_CONTEXT_TOKEN_INV returned by RRS service
ATREINT1.
.
Abend DC3 with reason 000C0006 is a generic abend.  For it to
be a match for this problem, minor codes C9C20020 and C9C21948
and message BBOO0021E must also be present.
.
There is a timing window when a few timeouts occur very close
together for requests that are running under transactions.  This
timing window can cause the controller region to abend this way.
This apar will close that window.
Local fix Problem summary
****************************************************************
* USERS AFFECTED: All users of WebSphere Application Server    *
*                 V5.0 for z/OS                                *
****************************************************************
* PROBLEM DESCRIPTION: ABENDSDC3/ABENDDC3 reason code 000C0006 *
*                      causes the controller to terminate.     *
*                      The following message is also seen in   *
*                      the controller job output:              *
*                                                              *
*                      Trace: xxxx/xx/xx yy:yy:yy.yyy 0000000A *
*                       Description: Log Boss/390 Error        *
*                       from filename: ./bbottmcr.cpp          *
*                       at line: 3492                          *
*                       error message: BBOO0021E Assertion     *
*                                      failed:  ctl_p != NULL  *
****************************************************************
* RECOMMENDATION:                                              *
****************************************************************
The controller abends with abend code DC3 reason code
000C0006.  The transaction service was attempting to look up
a transaction which had just been started in a servant.
An MDB dispatch in that same servant had just timed out
and caused the servant to abend.  When the servant
abended, the transaction which had just started became invalid,
and was deleted by another thread.  When the first thread
attempted to look up the transaction, it was not found and the
controller was terminated.

The following message shows the detail of the failed lookup:
Trace: xxxx/xx/xx yy:yy:yy.yyy 01 t=zzzzzz c=UNK key=S2
 Description: Log Boss/390 Error
 from filename: ./bbottmcr.cpp
 at line: 3492
 error message: BBOO0021E Assertion failed:  ctl_p != NULL
Problem conclusion
Code was added in the controller to handle the case when a
transaction times out and is deleted during begin processing.

The following publication was revised as a result
of APAR PQ98243:
________________________________________________________________
WebSphere Application Server V5 for z/OS
Messages and Codes
GA22-7915-00
_______________________________________________________________
This APAR requires changes to documentation.

NOTE: Periodically, we refresh the documentation on our
Web site, so the changes might have been made before you
read this text. To access the latest on-line
documentation, go to the product library page at:

www.ibm.com/software/webservers/appserv/zos_os390/library.html
________________________________________________________________
Chapter 3, pg. 154  New Minor Code

C9C20958

Explanation:  The transaction manager could not find a
              reference to a recently created transaction in
              the active GTID map.  This can happen when a
              server region initiated transaction completes
              during asynchronous processing.
User Response: Check the message and error logs for information
               relating to a transaction timeout as a timeout
               is the most likely cause of this error.
________________________________________________________________

APAR PQ98243 is associated with SERVICE LEVEL W502022 of
WebSphere Application Server V5.0 for z/OS.
Temporary fix Comments
APAR information
APAR number PQ98243
Reported component name WEBSPHERE FOR Z
Reported component ID 5655I3500
Reported release 500
Status CLOSED PER
PE NoPE
HIPER NoHIPER
Special Attention NoSpecatt
Submitted date 2004-12-09
Closed date 2005-01-21
Last modified date 2005-02-01

APAR is sysrouted FROM one or more of the following:

APAR is sysrouted TO one or more of the following:
PQ98247

Modules/Macros
BBOUBINF          

Publications Referenced

Fix information
Fixed component name WEBSPHERE FOR Z
Fixed component ID 5655I3500

Applicable component levels
R500 PSY UQ97067    UP05/01/26 P F501

  Fix is available
Select the PTF appropriate for your component level. You will be required to sign in. Distribution on physical media is not available in all countries.


Document Information


Current web document: swg1PQ98243.html
Product categories: Software > Application Servers > Distributed Application & Web Servers > WebSphere Application Server for z/OS
Operating system(s):
Software version: 500
Software edition:
Reference #: PQ98243
IBM Group: Software Group
Modified date: Feb 1, 2005