Fix (APAR): PK93653 Status: Fix Release: 6.1.0.29,6.1.0.27,6.1.0.25,6.0.2.39,6.0.2.37,6.0.2.35 Operating System: AIX,HP-UX,i5/OS,Linux,Solaris,Windows Supersedes Fixes: CMVC Defect: PK93653 Byte size of APAR: 207764 Date: 2010-08-25 Abstract: A server's ORB ListenerThread can become hung indefinitely when an SSL handshake with a client does not complete successfully. Description/symptom of problem: PK93653 resolves the following problem: PROBLEM SUMMARY When a client (or server acting as a client) is attempting to make a new secure connection to the WebSphere Application Server, an SSL handshake must ensue between the client and server. An Application Server's ORB SSL Listener Thread can become hung, no longer accepting new incoming connections when the SSL handshake does not complete successfully. On the server side, the Listener Thread will have the following call stack: "LT=1:P=817687:O=0:port=9409" (TID:0x45B37100, sys_thread_t:0x455DD178, state:R, native ID:0x00001558) prio=5 at java/net/SocketInputStream.socketRead0 at java/net/SocketInputStream.read(SocketInputStream.java:155 at com/ibm/jsse2/a.a(a.java:217 at com/ibm/jsse2/a.a(a.java:37) at com/ibm/jsse2/jc.a(jc.java:294 at com/ibm/jsse2/jc.g(jc.java:203) at com/ibm/jsse2/jc.a(jc.java:97) at com/ibm/jsse2/jc.getSession(jc.java:133) at com/ibm/ws/security/orbssl/WSSSLServerSocketFactoryImpl. getPeerCertificateChain at com/ibm/ws/orbimpl/transport/WSSSLTransportConnection$1.run at com/ibm/ws/security/util/AccessController.doPrivileged at com/ibm/ws/orbimpl/transport/WSSSLTransportConnection. at com/ibm/ws/orbimpl/transport/WSTransport.createTransportConnecti on at com/ibm/rmi/transport/ListenerThread.run It is possible that the client thread attempting the secure connection can also become hung, with startHandshake() in the call stack. Local Fix/Workaround: Recycle the Application Server whose Listener Thread is hung. USERS AFFECTED: All users of IBM WebSphere Application Server Network Deployment edition in v6.0.2, v6.1 and v7.0. PROBLEM DESCRIPTION: A server's ORB ListenerThread can become hung indefinitely when an SSL handshake with a client does not complete successfully. RECOMMENDATION: None PROBLEM CONCLUSION: A new ORB custom property "com.ibm.ws.orb.transport.SSLHandshakeTimeout" has been created to set a timeout on the reading of SSL handshake-related messages, thus preventing the ListenerThread hang should a message not be received by the timeout period specified in the property. You can add the ORB property to set a timeout for any JVM process you want, for example to add the ORB custom property for application server server1, go to the administrative console-->Servers-->Application Servers-->server1-->ORB service-->Custom Propertyies panel, and click New to add this new custom property: name=com.ibm.ws.orb.transport.SSLHandshakeTimeout value should be any integer bigger than 0, the measurement is milliseconds. If the property is not added, or not set to be a valid positive integer, the timeout will be default to 10000ms which means timeout in 10 seconds. The fix for this APAR is currently targeted for inclusion in fix packs 6.0.2.41, 6.1.0.31 and 7.0.0.9. Please refer to the Recommended Updates page for delivery information: http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980 Directions to apply fix: Fix applies to Editions: Release 6 _X_ Application Server (Express or BASE) _X_ Network Deployment (ND) __ WebSphere Business Integration Server Foundation (WBISF) __ Edge Components __ Developer __ Extended Deployment (XD) Install Fix to: Method: _X_ Application Server Nodes _X_ Deployment Manager Nodes __X Both NOTE: The user must: * Have Administrative rights in Windows, or be the Actual Root User in a UNIX environments. * Logged in with the same authority level when unpacking a fix, fix pack or refresh pack. * Be at V6.1.0.13 or newer of the Update Installer. Certain iFixes may require a newer version of the Update Installer and the Update Installer will inform you during the installation process if a newer version is required. This can be checked by reviewing the level of the Update Installer in file /updateinstaller/version.txt. The Update Installer can be downloaded from the following link: http://www.ibm.com/support/docview.wss?rs=180&uid=swg21205991 For detailed instructions to Extract the Update Installer see the following Technote: http://www.ibm.com/support/docview.wss?rs=180&uid=swg21205400 Note that there are two different methods for delivering iFixes, depending on the contents. The fix may be delivered either as a single file with a .pak extension (such as 6.1.0.11-WS-WAS-IFPK12345.pak) or a single file with a .zip extension (such as 6.1.0.11-WS-WAS-IFPK12345.zip) which then contains one or more files with a .pak extension. 1) If your iFix is delivered as a single file with a .pak extension, Copy the .pak file directly to the maintenance directory. If your iFix is delivered as a single file with a .zip extension, unzip the file into the maintenance directory. 2) Shutdown WebSphere Manually execute setupCmdLine.bat in Windows or . ./setupCmdLine.sh in Unix from the WebSphere instance that maintenance is being applied to. 3) Launch Update Installer and click the Next button on the Welcome page. 4) Enter the directory path of the installation location of the WebSphere product you want to update, and click the Next button. 5) Select the "Install maintenance package" operation and click the Next button. 6) Enter the directory path of your maintenance directory where you have the maintenance packages (.pak files) and click the Next button. 7) The Available Maintenance Package to Install page should list all maintenance packages (.pak files) that it finds in the directory path provided in the previous step. The Update Installer will select the correct maintenance packages based on your system configuration and will not allow an invalid combination to be installed. Please keep the Update Installer recommendations and click the Next button and continue with the installation of the maintenance package. 8) Please note that in the future, if a Feature Pack is installed or uninstalled, a different set of iFixes will be needed. Use the Update Installer again at that time, with the maintenance directory location where these maintenance packages are stored, to determine the required interim fixes for the new WebSphere and Feature Pack(s) combination. 9) The maintenance packages could have one of a set of names, and these names will help determine which maintenance package you need to install. The APAR name (PKxxxxx) should appear as part of the filename. Between the APAR number and the .pak extension there will be 0 to 2 characters added. The table below indicates the usage of each of the maintenance packages with respect to which Feature Packs, if any, are installed. |.pak File Names |No Feature Packs|EJB3 Only|WebServices Only| Both | |6.1.0.x-WS-WAS-IFPK12345 | X | X | X | X | |6.1.0.x-WS-WAS-IFPK12345C | X | | | | |6.1.0.x-WS-WAS-IFPK12345CE | | X | X | X | |6.1.0.x-WS-WAS-IFPK12345W | X | X | | | |6.1.0.x-WS-WAS-IFPK12345WE | | | X | X | |6.1.0.x-WS-WAS-IFPK12345E | X | | X | | |6.1.0.x-WS-WAS-IFPK12345EE | | X | | X | 10) Restart WebSphere Directions to remove fix: NOTE: * The user must have Administrative rights in Windows, or be the Actual Root User in a UNIX environments. * FIXES MUST BE REMOVED IN THE ORDER THEY WERE APPLIED * DO NOT REMOVE A FIX UNLESS ALL FIXES APPLIED AFTER IT HAVE FIRST BEEN REMOVED * YOU MAY REAPPLY ANY REMOVED FIX Example: If your system has fix1, fix2, and fix3 applied in that order and fix2 is to be removed, fix3 must be removed first, fix2 removed, and fix3 re-applied. 1) Shutdown WebSphere Application Server. Manually execute setupCmdLine.bat in Windows or . ./setupCmdLine.sh in Unix from the WebSphere instance that uninstall is being run against. 2) Start Update Installer 3) Enter the installation location of the WebSphere product you want to remove the fix. 4) Select "Uninstall maintenance package" operation. 5) Select file name of the maintenance package to uninstall (for example, 6.1.0.11-WS-WAS-IFPK12345.pak). 6) Verify the Uninstallation Summary and continue with the uninstall. 7) Restart WebSphere Directions to re-apply fix: 1) Shutdown WebSphere Application Server. 2) Follow the Fix instructions to apply the fix. 3) Restart WebSphere Application Server. Additional Information: