PQ80525: DEADLOCK causes jvm hang when the alarm thread, doing orphan timeout, and another thread close connection at the same time.

 A fix is available

4.0.2-4.0.7: Component cumulative Connection Manager fix



APAR status
Closed as unreproducible.

Error description
The external symptom is that the AppServer eventually stops
doing any processing that involves connections to a data source.
.
The javacores show that the Alarm thread is in orphan timeout
processing, such as this java call stack does:
"Alarm:5" (TID:0x406AEA8, sys_thread_t:0x148D6AA8, state:CW,
native ID:0x4CC) prio=5
 at com.ibm.ejs.cm.proxy.ConnectionProxy.connectionOrphaned
(ConnectionProxy.java:162)
 at com.ibm.ejs.cm.pool.ConnectO.fireOrphanTimeoutEvent
(ConnectO.java:2691)
 at com.ibm.ejs.cm.pool.ConnectO.checkForOrphan
(ConnectO.java:1253)
 at com.ibm.ejs.cm.pool.ConnectionPool.alarm
(ConnectionPool.java:1367)
.
Another thread will have a call stack like this one:
"Servlet.Engine.Transports:6" (TID:0x666DC80,
sys_thread_t:0x13C630D8, state:CW, native ID:0x924) prio=5
 at com.ibm.ejs.cm.pool.ConnectO.destroy(ConnectO.java:1300)
 at com.ibm.ejs.cm.pool.ConnectionPool.destroyConnection
(ConnectionPool.java::1059)
 at com.ibm.ejs.cm.pool.ConnectO.translateException
(ConnectO.java:1989)
 at com.ibm.ejs.cm.proxy.Proxy.translateException
(Proxy.java(Compiled Code))
 at com.ibm.ejs.cm.proxy.StatementProxy.close
(StatementProxy.java(Compiled Code))
.
The monitor information in the javacore will show that each of
these threads is blocked on the other.  You will likely also see
a number of the rest of the Servlet.Engine.Transports threads
blocked on com.ibm.ejs.cm.pool.ConnectionPool@6D009A8/6D009B0,
that is on the ConnectionPool synchronization lock.
   Here is what the monitor information is likely to look like.
Here is the monitor for the thread doing the destroyConnection.
The Alarm thread is blocked on it.
sys_mon_t:0x148DFCE0 infl_mon_t: 0x00000000:
com.ibm.ejs.cm.proxy.OracleCallableStatementProxy@2DD7940/2DD794
8:
Flat locked by thread ident 0x23, entry count 1
 Waiting to be notified:
     "Alarm:5" (0x148D6AA8)
.
Here is the monitor that the Alarm thread holds.  The
Servlet.Engine.Tranports thread is blocked on it.
sys_mon_t:0x13AB8CF8 infl_mon_t: 0x00000000:
    com.ibm.ejs.cm.pool.ConnectO@417AE58/417AE60: Flat locked by
thread ident 0x2D, entry count 1 ( ident 0x2D "Alarm:5")
 Waiting to be notified:
     "Servlet.Engine.Transports:6" (0x13C630D8)
Local fix
This is fixed in WAS 4.0.5 and later and is also included in the
latest available Connection Manager cumulative fix.
This defect 142913.
Problem summary
****************************************************************
* USERS AFFECTED: All WebSphere Application Server users       *
*                 of connection pooling.                       *
****************************************************************
* PROBLEM DESCRIPTION: A deadlock could occur between          *
*                      closing a connection and Orphaning      *
*                      a connection.                           *
****************************************************************
* RECOMMENDATION:                                              *
****************************************************************
If the Orphan timer went off at the exact moment during
a close() operation, there was small window where a deadlock
was happening.  This was fixed in internal defect 142913
Problem conclusion Temporary fix Comments
This is a duplicate of internal defect 142913.  The
problem symptom is a deadlock between the Orphaning and
closing a connection.  This was fixed in the base of 4.0.5,
it is also in all follow-on releases.
APAR information
APAR number PQ80525
Reported component name WEBSPHERE AE NT
Reported component ID 5630A2201
Reported release 400
Status CLOSED UR5
PE NoPE
HIPER NoHIPER
Submitted date 2003-11-05
Closed date 2003-11-10
Last modified date 2003-11-10

APAR is sysrouted FROM one or more of the following:

APAR is sysrouted TO one or more of the following:

Modules/Macros

SRLS

Fix information

Applicable component levels


Document Information


Product categories: Software > Application Servers > Distributed Application & Web Servers > WebSphere Application Server > General
Operating system(s):
Software version: 400
Software edition:
Reference #: PQ80525
IBM Group: Software Group
Modified date: Nov 10, 2003