Fix (APAR): PM82550 Status: Fix Release: 7.0.0.27,7.0.0.25 Operating System: AIX,HP-UX,IBM i,Linux,Solaris,Windows,z/OS Supersedes Fixes: PM78427 CMVC Defect: PM82550 Byte size of APAR: 55624 Date: 2013-03-27 Abstract: Deak lock in SIP cotnainer Description/symptom of problem: PM82550 resolves the following problem: ERROR DESCRIPTION: WebSphere logs show deadlock in the SIP Container threads. The application server may crash with an out of memory error (OOM) and many hung threads were detected in the SIP Container. Deadlock detected !!! --------------------- Thread "Thread-60" (0xB1689000) is waiting for: sys_mon_t:0xAD657FB8 infl_mon_t: 0xAD657FF0: java/lang/Object@0xBC187CD8/0xBC187CE4: which is owned by: Thread "SipContainer : 1" (0xAE298D00) which is waiting for: sys_mon_t:0xAD658108 infl_mon_t: 0xAD658140: com/ibm/ws/sip/container/proxy/StatefullProxy@0xBC1976A0/0xBC197 6AC: which is owned by: Thread "Thread-60" (0xB1689000) "Thread-60" J9VMThread:0xB1689000, j9thread_t:0x09926EE0, java/lang/Thread:0xB886D978, state:B, prio=5 (native thread ID:0x62E9, native priority:0x5, native policy:UNKNOWN) (native stack address range from:0xABB7A000, to:0xABBBB000, size:0x41000) Java callstack: at com/ibm/ws/sip/container/tu/TransactionUserWrapper.addToTransact at com/ibm/ws/sip/container/proxy/StatefullProxy.onSendingRequest at com/ibm/ws/sip/container/proxy/ProxyBranchImpl.onSendingRequest at com/ibm/ws/sip/container/transaction/ClientTransaction.sendReque at com/ibm/ws/sip/container/servlets/OutgoingSipServletRequest.send at com/ibm/ws/sip/container/proxy/BranchManager.cancelRequest at com/ibm/ws/sip/container/proxy/ProxyBranchImpl.cancel at com/ibm/ws/sip/container/proxy/ProxyBranchImpl.cancel at com/ibm/ws/sip/container/proxy/ProxyBranchImpl.executeTimeOut at com/ibm/ws/sip/container/proxy/ProxyBranchImpl.proxyTimedOut at com/ibm/ws/sip/container/proxy/BranchManager.timeoutAllChildBran at com/ibm/ws/sip/container/proxy/StatefullProxy.proxyTimeout at com/ibm/ws/sip/container/proxy/ProxyTimer.invoke at com/ibm/ws/sip/container/timer/SipTimerTask.run at java/util/Timer$TimerImpl.run "SipContainer : 1" J9VMThread:0xAE298D00, j9thread_t:0xAF0AFF18, java/lang/Thread:0xC40BA7A0, state:B, prio=5 (native thread ID:0x624E, native priority:0x5, native policy:UNKNOWN) (native stack address range from:0xAC1BB000, to:0xAC1FC000, size:0x41000) Java callstack: at com/ibm/ws/sip/container/servlets/OutgoingSipServletResponse.con at com/ibm/ws/sip/container/servlets/OutgoingSipServletResponse.sen at com/ibm/ws/sip/container/tu/TransactionUserImpl.terminateUnderly at com/ibm/ws/sip/container/tu/TransactionUserImpl.invalidateTU at com/ibm/ws/sip/container/tu/TransactionUserWrapper.invalidateTU at com/ibm/ws/sip/container/tu/TransactionUserWrapper.invalidateTU at com/ibm/ws/sip/container/servlets/SipApplicationSessionImpl.dest at com/ibm/ws/sip/container/servlets/SipApplicationSessionImpl.inva at com/ibm/ws/sip/container/servlets/WASXSipApplicationSessionImpl. at com/ibm/ws/sip/container/timer/AppSessionTimerListener.invokeExp at com/ibm/ws/sip/container/timer/ExpirationTimer.run at com/ibm/ws/sip/container/was/SignalingEndOfTask.run at com/ibm/ws/util/ThreadPool$Worker.run LOCAL FIX: NONE PROBLEM SUMMARY USERS AFFECTED: All users of IBM WebSphere Application Server Session Initiation Protocol (SIP) PROBLEM DESCRIPTION: Deak lock in SIP cotnainer RECOMMENDATION: None When session timout fired at the same time as Proxy timeout - the events are not running on the same thread. PROBLEM CONCLUSION: Code has been updated to correct the deadlock. The fix for this APAR is currently targeted for inclusion in fix packs 7.0.0.25, 8.0.0.7 and the fix pack following 8.5.0.2. Please refer to the Recommended Updates page for delivery information: http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980 Directions to apply fix: Fix applies to Editions: Release 7.0 X Application Server (Express or BASE) X Network Deployment (ND) __ Edge Components __ Developer Install Fix to all WebSphere installations unless special instructions are included below. Special Instructions: None NOTE: The user must: * Logged in with the same authority level when unpacking a fix, fix pack or refresh pack. * Be at V7.0.0.0 or newer of the Update Installer. Certain iFixes may require a newer version of the Update Installer and the Update Installer will inform you during the installation process if a newer version is required. This can be checked by reviewing the level of the Update Installer in file /updateinstaller/version.txt. The Update Installer can be downloaded from the following link: http://www.ibm.com/support/docview.wss?rs=180&uid=swg21205991 1) If your iFix is delivered as a single file with a .pak extension, Copy the .pak file directly to the maintenance directory. If your iFix is delivered as a single file with a .zip extension, unzip the file into the maintenance directory. 2) Shutdown WebSphere Application Server. Manually execute setupCmdLine.bat in Windows or . ./setupCmdLine.sh in Unix from the WebSphere instance that maintenance is being applied to. 3) For IBM i users, use the update command to install and uninstall the interim fix. The IBM Information Center can provide additional details, if needed, on the use of this command. http://www14.software.ibm.com/webapp/wsbroker/redirect?version=compass&product=was-nd-iseries&topic=rins_update. For non-IBM i users, launch the Update Installer and click the Next button on the Welcome page. 4) Enter the directory path of the installation location of the WebSphere product you want to update, and click the Next button. 5) Select the "Install maintenance package" operation and click the Next button. 6) Enter the directory path of your maintenance directory where you have the maintenance packages (.pak files) and click the Next button. 7) The Available Maintenance Package to Install page should list all maintenance packages (.pak files) that it finds in the directory path provided in the previous step. The Update Installer will select the correct maintenance packages based on your system configuration and will not allow an invalid combination to be installed. Please keep the Update Installer recommendations and click the Next button and continue with the installation of the maintenance package. To determine why some maintenance packages have been identified as not applicable, see description in log found in /logs/tmp*/updatelogs.txt 8) For all platforms except Windows. In pre-install summary panel, use the "verify permission" feature to verify the user has permissions to apply updates to files associated with the selected maintenance. Correct any file permissions before clicking next to start the install. 9) Restart WebSphere Application Server. Directions to remove fix: NOTE: * The user must have Administrative rights in Windows, or be the Actual Root User in a UNIX environments. * FIXES MUST BE REMOVED IN THE ORDER THEY WERE APPLIED * DO NOT REMOVE A FIX UNLESS ALL FIXES APPLIED AFTER IT HAVE FIRST BEEN REMOVED * YOU MAY REAPPLY ANY REMOVED FIX Example: If your system has fix1, fix2, and fix3 applied in that order and fix2 is to be removed, fix3 must be removed first, fix2 removed, and fix3 re-applied. 1) Shutdown WebSphere Application Server. Manually execute setupCmdLine.bat in Windows or . ./setupCmdLine.sh in Unix from the WebSphere instance that uninstall is being run against. 2) Start Update Installer 3) Enter the installation location of the WebSphere product you want to remove the fix. 4) Select "Uninstall maintenance package" operation. 5) Enter the file name of the maintenance package to uninstall (PKxxxxx.pak). 6) UnInstall maintenance package. 7) Restart WebSphere Directions to re-apply fix: 1) Shutdown WebSphere Application Server. 2) Follow the Fix instructions to apply the fix. 3) Restart WebSphere Application Server. Additional Information: