PK02851: ABENDEC3 RC=04130007 SYNCHRONIZEDJAVA METHOD WEBAPPSERVLETMANAGER.LOADSERVLET HELD BY TCB WITH BBOOWSEM

 A fix is available

Obtain the fix for this APAR



APAR status
Closed as program error.

Error description
WebSphere Application server receives ABENDEC3 RC=04130007.
The application server was in a slow state, leading up to the
ABEND.  This is a HTTPRequest timeout, which is controlled by
the 'protocol_http_timeout_output' timer value.
.
In this timeout condition, the thread that received the ABENDEC3
in the servant region had the following stack Trace:
Function
--------
CEEOPCW
pthread_cond_wait
condWait
sysMonitorWait
lkMonitorEnter
@@GETFN
com/ibm/ws/webcontainer/webapp/WebAppServletManager.loadServlet
same as above/WebAppServletManager.getServletReference
same as above/WebApp.getServletReference
same as above/WebAppRequestDispatcherInfo.calculateInfo
same as above/WebAppRequestDispatcherInfo.<init>
same as above/WebApp.getRequestDispatcher
same as above/WebApp.getRequestDispatcher
com/appcode/servlet/inquiry/BaseInquiry.processRequest
com/appcode/servlet/BaseServlet.doPost
com/appcode/servlet/BaseServlet.doGet
javax/servlet/http/HttpServlet.service
javax/servlet/http/HttpServlet.service
com/ibm/ws/webcontainer/servlet/StrictServletInstance.doService
same as above/StrictLifecycleServlet._service
same as above/ServicingServletState.service
same as above/StrictLifecycleServlet.service
same as above/ServletInstance.service
same as above/ValidServletReferenceState.dispatch
same as above/ServletInstanceReference.dispatch
/webapp/WebAppRequestDispatcher.handleWebAppDispatch
/webapp/WebAppRequestDispatcher.dispatch
/webapp/WebAppRequestDispatcher.forward
com/ibm/ws/webcontainer/srt/WebAppInvoker.doForward
.
The method at the top of the stack:
com/ibm/ws/webcontainer/webapp/WebAppServletManager.loadServlet
is a mmipSelectInvokeSynchronizedJavaMethod and is 'waiting'
on the java lock on this method to execute.
.
Using jformat, the 'DIS LS' command, we can see that another TCB
is holding this 'lock'
.
The stack trace of the TCB holding the lock is:
Function
--------
BBOOWSEM
SR_ExecutionThread::RemoveAndProcessWork(ThreadCleanUp*,TCB*)
SR_ExecutionRoutine
(unknown)
.
In addition, in the CTRACE data from the SVC Dump, we see that
the TCB which is HOLDING the lock, has issued:
BBOO0164W Function vfprintf failed with RC=-1. System
determined REASON=EDC5113I Bad file descriptor.
(errno2=0x05710034)
from bboumsg, line 623 (W502014).
We also see:
EXCEPTN= CORBA::INTERNAL  (Throwing the exception)
MINORCD= 0xC9C2040F
CHR_STR= ./bboumsg.cpp
LINENUMBER= 624
in the CTRACE, followed by:
EXCEPTN= CORBA::INTERNAL       (Catching the exception)
MINORCD= 0xC9C2040F
CHR_STR= ./bbooejsb.cpp
LINENUMBER= 1215
.
Again, the above is on the thread that is HOLDING the lock on
the synchronizedmethod.
Local fix Problem summary
****************************************************************
* USERS AFFECTED: All users of WebSphere Application Server    *
*                 V5.0 for z/OS                                *
****************************************************************
* PROBLEM DESCRIPTION: WebSphere Application Server V5 for     *
*                      z/OS receives ABENDEC3/ABENDSEC3        *
*                      RC=04130007 after a servant hung.       *
****************************************************************
* RECOMMENDATION:                                              *
****************************************************************
A number of servant threads were waiting on a serialized block
that another thread was holding. One of the waiting threads had
com/ibm/ws/webcontainer/webapp/WebAppServletManager.loadServlet
a mmipSelectInvokeSynchronizedJavaMethod, and is 'waiting'
on the java lock on this method to execute.

The stack trace of the TCB holding the lock is:
Function
--------
BBOOWSEM
SR_ExecutionThread::RemoveAndProcessWork(ThreadCleanUp*,TCB*)
SR_ExecutionRoutine

The entire call stack including BBOOWSEM is native code.
However, this thread is holding Java locks.  A trace record
found in the dump showed a native error under a JNI
call.  A SystemException was thrown as a result.  The JNI
routine did not have a catch for the native throw. The LE
runtime catch processing found the previous catch block
to be a catch block setup on the thread prior to dispatching
the request in Java.  It gave control to this catch block
which bypassed all Java catches and end of block cleanup.
Problem conclusion
Code has been added in the JNI routines to catch native
exceptions and generate Java exceptions to return to the Java
callers.

APAR PK02851 requires changes to documentation.
The following publication GA22-7915-01
was revised as a result of APAR PK02851.
________________________________________________________________
WebSphere Application Server V5.0 for z/OS
Messages and Codes
GA22-7915-01
_______________________________________________________________
This APAR requires changes to documentation.

NOTE: Periodically, we refresh the documentation on our
Web site, so the changes might have been made before you
read this text. To access the latest on-line
documentation, go to the product library page at:

www.ibm.com/software/webservers/appserv/zos_os390/library.html

________________________________________________________________
Chapter 3, pg. 125 (new messages)
C9C2135A - C9C2137F
Explanation: IBM Internal Only
User Response: Contact the IBM Support Center.
________________________________________________________________
Chapter 3, pg. 188 (new minor code)
C9C2E44A
Explanation: The minor code indicates an out of memory
condition when a WebSphere module attempted to allocate
storage.
Suggested Action: See minor code C9C2E001.
________________________________________________________________
Chapter 3, pg. 78 (new message)
C9C20034 - C9C20035
Explanation: IBM Internal Only
User Response: Contact the IBM Support Center.
________________________________________________________________
Chapter 3, pg. 178 (new messages)
C9C24135 - C9C24153
Explanation: IBM Internal Only
User Response: Contact the IBM Support Center.
________________________________________________________________
Chapter 3, pg. 145 (new messages)
C9C2196E - C9C21973
Explanation: IBM Internal Only
User Response: Contact the IBM Support Center.
________________________________________________________________
Chapter 3, pg. 79 (new messages)
C9C20413 - C9C20418
Explanation: IBM Internal Only
User Response: Contact the IBM Support Center.

APAR PK02851 is associated with SERVICE LEVEL W502029 of
WebSphere Application Server V5.0 for z/OS.
Temporary fix Comments
APAR information
APAR number PK02851
Reported component name WEBSPHERE FOR Z
Reported component ID 5655I3500
Reported release 500
Status CLOSED PER
PE NoPE
HIPER NoHIPER
Special Attention YesSpecatt / CST
Submitted date 2005-03-21
Closed date 2005-04-29
Last modified date 2005-06-17

APAR is sysrouted FROM one or more of the following:

APAR is sysrouted TO one or more of the following:
PK04181 PK04182

Modules/Macros
BBOUBINF          

Publications Referenced
GA22791501        

Fix information
Fixed component name WEBSPHERE FOR Z
Fixed component ID 5655I3500

Applicable component levels
R500 PSY UK03008    UP05/05/05 P F505

  Fix is available
Select the PTF appropriate for your component level. You will be required to sign in. Distribution on physical media is not available in all countries.


Document Information


Current web document: swg1PK02851.html
Product categories: Software > Application Servers > Distributed Application & Web Servers > WebSphere Application Server for z/OS
Operating system(s):
Software version: 500
Software edition:
Reference #: PK02851
IBM Group: Software Group
Modified date: Jun 17, 2005