PK32749: TOSTRING METHOD HOLDING THE LOCK TO THE CONNECTION HANDLE

 Fixes are available

6.1.0.15 WebSphere Application Server V6.1 Fix Pack 15 for i5/OS
5.1.1.17: WebSphere Application Server V5.1.1 Cumulative Fix 17 for AIX
5.1.1.17: WebSphere Application Server V5.1.1 Cumulative Fix 17 for HP-UX
6.1.0.13 WebSphere Application Server V6.1 Fix Pack 13 for AIX
5.1.1.19: WebSphere Application Server V5.1.1 Cumulative Fix 19 for Linux
5.1.1.16: WebSphere Application Server V5.1.1 Cumulative Fix 16 for AIX
6.0.2.25: WebSphere Application Server V6.0.2 Fix Pack 25 for AIX platforms
5.1.1.18: WebSphere Application Server V5.1.1 Cumulative Fix 18 for AIX
5.1.1.18: WebSphere Application Server V5.1.1 Cumulative Fix 18 for HP-UX
6.0.2.27: WebSphere Application Server V6.0.2 Fix Pack 27 for HP-UX platforms
6.1.0.15 WebSphere Application Server V6.1 Fix Pack 15 for AIX
5.1.1.18: WebSphere Application Server V5.1.1 Cumulative Fix 18 for Solaris
5.1.1.18: WebSphere Application Server V5.1.1 Cumulative Fix 18 for Windows
6.1.0.15: WebSphere Application Server V6.1 Fix Pack 15 for HP-UX
6.1.0.15: WebSphere Application Server V6.1 Fix Pack 15 for Windows
6.0.2.27: WebSphere Application Server V6.0.2 Fix Pack 27 for OS/400 platform
6.1.0.13: WebSphere Application Server V6.1 Fix Pack 13 for Windows
6.1.0.17 WebSphere Application Server V6.1 Fix Pack 17 for i5/OS
6.1.0.13: WebSphere Application Server V6.1 Fix Pack 13 for i5/OS
6.1.0.13: WebSphere Application Server V6.1 Fix Pack 13 for HP-UX
6.0.2.27: WebSphere Application Server V6.0.2 Fix Pack 27 for Solaris
6.0.2.27: WebSphere Application Server V6.0.2 Fix Pack 27 for Windows platforms
6.1.0.17: WebSphere Application Server V6.1 Fix Pack 17 for Linux
6.0.2.27: WebSphere Application Server V6.0.2 Fix Pack 27 for AIX platforms
6.0.2.25: WebSphere Application Server V6.0.2 Fix Pack 25 for HP-UX platforms
6.1.0.17: WebSphere Application Server V6.1 Fix Pack 17 for Solaris
6.0.2.23: WebSphere Application Server V6.0.2 Fix Pack 23 for HP-UX platforms
6.1.0.17: WebSphere Application Server V6.1 Fix Pack 17 for HP-UX
6.1.0.17: WebSphere Application Server V6.1 Fix Pack 17 for Windows
6.1.0.17 WebSphere Application Server V6.1 Fix Pack 17 for AIX
6.1.0.13: WebSphere Application Server V6.1 Fix Pack 13 for Solaris
5.1.1.18: WebSphere Application Server V5.1.1 Cumulative Fix 18 for Linux
6.1.0.15: WebSphere Application Server V6.1 Fix Pack 15 for Linux
5.1.1.17: WebSphere Application Server V5.1.1 Cumulative Fix 17 for Linux
5.1.1.17: WebSphere Application Server V5.1.1 Cumulative Fix 17 for Solaris
5.1.1.17: WebSphere Application Server V5.1.1 Cumulative Fix 17 for Windows
6.0.2.23: WebSphere Application Server V6.0.2 Fix Pack 23 for AIX platforms
6.1.0.15: WebSphere Application Server V6.1 Fix Pack 15 for Solaris
5.1.1.19: WebSphere Application Server V5.1.1 Cumulative Fix 19 for AIX
6.0.2.29: WebSphere Application Server V6.0.2 Fix Pack 29 for AIX platforms
6.0.2.29: WebSphere Application Server V6.0.2 Fix Pack 29 for HP-UX platforms
6.0.2.29: WebSphere Application Server V6.0.2 Fix Pack 29 for Linux platforms
V6.0.2: Java SDK 1.4.2 SR11 Cumulative Fix for IBM WebSphere Application Server
6.0.2.29: WebSphere Application Server V6.0.2 Fix Pack 29 for Solaris
6.0.2.29: WebSphere Application Server V6.0.2 Fix Pack 29 for Windows platforms
6.0.2.27: WebSphere Application Server V6.0.2 Fix Pack 27 for Linux platforms
5.1.1.19: WebSphere Application Server V5.1.1 Cumulative Fix 19 for Windows
5.1.1.16: WebSphere Application Server V5.1.1 Cumulative Fix 16 for Solaris
5.1.1.16: WebSphere Application Server V5.1.1 Cumulative Fix 16 for Windows
5.1.1.14: WebSphere Application Server V5.1.1 Cumulative Fix 14 for Solaris
6.1.0.5: WebSphere Application Server V6.1.0 Fix Pack 5 for Solaris
6.1.0.7 WebSphere Application Server V6.1 Fix Pack 7 for AIX
6.1.0.5: WebSphere Application Server V6.1.0 Fix Pack 5 for AIX
6.0.2.25: WebSphere Application Server V6.0.2 Fix Pack 25 for Linux platforms
6.0.2.25: WebSphere Application Server V6.0.2 Fix Pack 25 for Solaris
6.0.2.25: WebSphere Application Server V6.0.2 Fix Pack 25 for Windows platforms
5.1.1.15: WebSphere Application Server V5.1.1 Cumulative Fix 15 for Solaris
6.1.0.5: WebSphere Application Server V6.1.0 Fix Pack 5 for Windows
6.1.0.5: WebSphere Application Server V6.1.0 Fix Pack 5 for HP-UX
6.0.2.19: WebSphere Application Server V6.0.2 Fix Pack 19 for AIX platforms
6.1.0.7: WebSphere Application Server V6.1 Fix Pack 7 for i5/OS
6.1.0.5: WebSphere Application Server V6.1.0 Fix Pack 5 for i5/OS
5.1.1.14: WebSphere Application Server V5.1.1 Cumulative Fix 14 for AIX
5.1.1.14: WebSphere Application Server V5.1.1 Cumulative Fix 14 for Linux
5.1.1.14: WebSphere Application Server V5.1.1 Cumulative Fix 14 for Windows
5.1.1.15: WebSphere Application Server V5.1.1 Cumulative Fix 15 for Windows
6.1.0.9 WebSphere Application Server V6.1 Fix Pack 9 for AIX
6.1.0.9: WebSphere Application Server V6.1 Fix Pack 9 for i5/OS
6.1.0.7: WebSphere Application Server V6.1 Fix Pack 7 for Windows
5.1.1.14: WebSphere Application Server V5.1.1 Cumulative Fix 14 for HP-UX
6.0.2.19: WebSphere Application Server V6.0.2 Fix Pack 19 for HP-UX platforms
6.0.2.19: WebSphere Application Server V6.0.2 Fix Pack 19 for Windows platforms
6.1.0.9: WebSphere Application Server V6.1 Fix Pack 9 for HP-UX
6.0.2.19: WebSphere Application Server V6.0.2 Fix Pack 19 for OS/400 platform
5.1.1.15: WebSphere Application Server V5.1.1 Cumulative Fix 15 for AIX
5.1.1.15: WebSphere Application Server V5.1.1 Cumulative Fix 15 for HP-UX
6.0.2.21: WebSphere Application Server V6.0.2 Fix Pack 21 for HP-UX platforms
6.0.2.21: WebSphere Application Server V6.0.2 Fix Pack 21 for Linux platforms
6.1.0.5: WebSphere Application Server V6.1.0 Fix Pack 5 for Linux
6.1.0.7: WebSphere Application Server V6.1 Fix Pack 7 for HP-UX
6.1.0.9: WebSphere Application Server V6.1 Fix Pack 9 for Linux
6.1.0.9: WebSphere Application Server V6.1 Fix Pack 9 for Solaris
6.1.0.9: WebSphere Application Server V6.1 Fix Pack 9 for Windows
6.1.0.11: WebSphere Application Server V6.1 Fix Pack 11 for HP-UX
6.1.0.11: WebSphere Application Server V6.1 Fix Pack 11 for Windows
5.1.1.16: WebSphere Application Server V5.1.1 Cumulative Fix 16 for HP-UX
5.1.1.16: WebSphere Application Server V5.1.1 Cumulative Fix 16 for Linux
6.0.2.23: WebSphere Application Server V6.0.2 Fix Pack 23 for Windows platforms
6.0.2.23: WebSphere Application Server V6.0.2 Fix Pack 23 for Solaris
6.0.2.23: WebSphere Application Server V6.0.2 Fix Pack 23 for OS/400 platform
5.1.1.15: WebSphere Application Server V5.1.1 Cumulative Fix 15 for Linux
6.0.2.23: WebSphere Application Server V6.0.2 Fix Pack 23 for Linux platforms
6.0.2.21: WebSphere Application Server V6.0.2 Fix Pack 21 for AIX platforms
6.0.2.21: WebSphere Application Server V6.0.2 Fix Pack 21 for Windows platforms
6.1.0.7: WebSphere Application Server V6.1 Fix Pack 7 for Solaris
6.0.2.21: WebSphere Application Server V6.0.2 Fix Pack 21 for Solaris platforms
6.1.0.7: WebSphere Application Server V6.1 Fix Pack 7 for Linux
6.0.2.19: WebSphere Application Server V6.0.2 Fix Pack 19 for Solaris
6.1.0.11: WebSphere Application Server V6.1 Fix Pack 11 for Solaris
6.1.0.11: WebSphere Application Server V6.1 Fix Pack 11 for Linux
6.1.0.11: WebSphere Application Server V6.1 Fix Pack 11 for i5/OS
6.1.0.11 WebSphere Application Server V6.1 Fix Pack 11 for AIX
6.0.2.19: WebSphere Application Server V6.0.2 Fix Pack 19 for Linux platforms
6.1.0.13: WebSphere Application Server V6.1 Fix Pack 13 for Linux
5.1.1.19: WebSphere Application Server V5.1.1 Cumulative Fix 19 for HP-UX
6.1.0.19 WebSphere Application Server V6.1 Fix Pack 19 for AIX
6.1.0.19: WebSphere Application Server V6.1 Fix Pack 19 for HP-UX
6.1.0.19 WebSphere Application Server V6.1 Fix Pack 19 for i5/OS
6.1.0.19: WebSphere Application Server V6.1 Fix Pack 19 for Linux
6.1.0.19: WebSphere Application Server V6.1 Fix Pack 19 for Solaris
6.1.0.19: WebSphere Application Server V6.1 Fix Pack 19 for Windows
Java SDK 1.5 SR8 Cumulative Fix for WebSphere Application Server
6.1.0.21 WebSphere Application Server V6.1 Fix Pack 21 for AIX
6.0.2.31: WebSphere Application Server V6.0.2 Fix Pack 31 for AIX platforms
6.0.2.31: WebSphere Application Server V6.0.2 Fix Pack 31 for HP-UX platforms
6.0.2.31: WebSphere Application Server V6.0.2 Fix Pack 31 for OS/400 platform
6.0.2.31: WebSphere Application Server V6.0.2 Fix Pack 31 for Linux platforms
6.0.2.31: WebSphere Application Server V6.0.2 Fix Pack 31 for Solaris
6.0.2.31: WebSphere Application Server V6.0.2 Fix Pack 31 for Windows platforms
V6.0.2: Java SDK 1.4.2 SR11 Cumulative Fix for IBM WebSphere Application Server
6.1.0.21 WebSphere Application Server V6.1 Fix Pack 21 for i5/OS
6.1.0.21: WebSphere Application Server V6.1 Fix Pack 21 for Windows
6.1.0.21: WebSphere Application Server V6.1 Fix Pack 21 for Solaris
6.1.0.21: WebSphere Application Server V6.1 Fix Pack 21 for Linux
Java SDK 1.5 SR8 Cumulative Fix for WebSphere Application Server
6.1.0.21: WebSphere Application Server V6.1 Fix Pack 21 for HP-UX



APAR status
Closed as program error.

Error description
toString method holding the lock to the connection handle.

Scenario: In a stack it showed that toString in the
JMSManagedConnection class. This tries to access the
openConnectionHandles in order to interate through them.
However, the cleanup method desribed
below...

4XESTACKTRACE          at
com.ibm.ejs.jms.JMSManagedConnection.cleanup(JMSManagedConnectio
n.java:7
32)
4XESTACKTRACE          at
com.ibm.ejs.j2c.MCWrapper.cleanup(MCWrapper.java:1201)
4XESTACKTRACE          at
com.ibm.ejs.j2c.poolmanager.FreePool.cleanupAndDestroyMCWrapper(
FreePool
.java:497)


---> has the lock. The deadlock is occurring because further
down the stack for the toString method, see below


4XESTACKTRACE          at
com.ibm.ejs.jms.JMSManagedConnection.toString(JMSManagedConnecti
on.java(
Compiled Code))
4XESTACKTRACE          at
java.lang.String.valueOf(String.java(Compiled
Code))
4XESTACKTRACE          at
java.lang.StringBuffer.append(StringBuffer.java(Compiled Code))
4XESTACKTRACE          at
com.ibm.ejs.j2c.MCWrapper.toString(MCWrapper.java(Compiled
Code))
4XESTACKTRACE          at
com.ibm.ejs.j2c.ConnectionEventListener.connectionErrorOccurred(
Connecti
onEventListener.java:307)
4XESTACKTRACE          at
com.ibm.ejs.jms.JMSManagedConnection.connectionErrorOccurred(JMS
ManagedC
onnection.java:1141)
4XESTACKTRACE          at
com.ibm.ejs.jms.JMSConnectionHandle.markManagedConnectionAsStale
(JMSConn
ectionHandle.java:702)
4XESTACKTRACE          at
com.ibm.ejs.jms.JMSSessionHandle.markManagedObjectsAsStale(JMSSe
ssionHan
dle.java:1365)


---> it was ultimately called from the method
markManagedConnectionAsStale, which is synchronised, and so is
holding the lock to the connection handle which the other
thread needs access to.
Local fix
candidate to be fix in next released
Problem summary
****************************************************************
* USERS AFFECTED: Websphere Application Server users of JMS    *
****************************************************************
* PROBLEM DESCRIPTION: When using the Java Message Service     *
*                      (JMS) functionality provided with       *
*                      WebSphere Application Server a          *
*                      deadlock can occur if the connection    *
*                      between the application server and      *
*                      the JMS provider is broken.             *
****************************************************************
* RECOMMENDATION:                                              *
****************************************************************
The deadlock results in two application server threads getting
stuck, and the JMS connection that was broken is never
returned to the connection pool.

The deadlock is caused by a very small timing window in the
JMSManagedConnection class.
The cleanup method iterates through a SynchronizedSet called
openConnectionHandles (in a block synchronised on
openConnectionHandle). As part of this it tried to call the
invalidate method on one of these handles, but the invalidate
method is synchronised (it needs access to connection handle).
In another thread we are in the JMSManagedConnection.toString()
method. This tries to access the openConnectionHandles in order
to interate through them. However, the cleanup method desribed
above has the lock.
The deadlock is occurring because further down the stack for
the toString method(), it was ultimately called from the method
markManagedConnectionAsStale(), which is synchronised, and so
is holding the lock to the connection handle which the other
thread needs access to.
Problem conclusion
To resolve the problem, the toString() method will make a copy
of the openConnectionHandles SynchronizedSet, and use this.
This means that toString() will no longer needs to have a lock
on the openConnectionHandles object, therefore preventing the
deadlock.

The fix for this APAR is currently targeted for inclusion in
Cumulative Fix 14 for WebSphere Application Server Version
5.1.1, Fix Pack 19 for WebSphere Application Server Version
6.0.2 and Fix Pack 5 for WebSphere Application Server Version
6.1.  Please refer to the recommended updates page for
delivery information:


http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980
Temporary fix Comments
APAR information
APAR number PK32749
Reported component name WAS NETWRK DEPL
Reported component ID 5630A3601
Reported release 10A
Status CLOSED PER
PE NoPE
HIPER NoHIPER
Special Attention NoSpecatt
Submitted date 2006-10-11
Closed date 2006-10-20
Last modified date 2006-11-06

APAR is sysrouted FROM one or more of the following:

APAR is sysrouted TO one or more of the following:

Modules/Macros
MSGING          

Publications Referenced

Fix information
Fixed component name WAS NETWRK DEPL
Fixed component ID 5630A3601

Applicable component levels
R003 PSN    UP
R00A PSN    UP
R00H PSN    UP
R00I PSN    UP
R00P PSN    UP
R00S PSN    UP
R00W PSN    UP
R103 PSY    UP
R10A PSY    UP
R10H PSY    UP
R10I PSY    UP
R10P PSY    UP
R10S PSY    UP
R10W PSY    UP


Document Information


Product categories: Software > Application Servers > Distributed Application & Web Servers > WebSphere Application Server > General
Operating system(s):
Software version: 10A
Software edition:
Reference #: PK32749
IBM Group: Software Group
Modified date: Nov 6, 2006