APAR status
Closed as program error.
Error description
Running WSAS Enterprise Edition, the nodeagent
seems to hang. If a javacore is taken by issuing a "kill -3"
command, a deadlock in security threads is seen.
The locked threads in the javacore and the security problem
seems to match APAR
PQ83910 for Base and ND.
SystemOut.log for the nodeagent shows these errors:
===================================================
[10/31/04 5:31:17:413 EST] 1c1aa27c LdapRegistryI E SECJ0336E:
Authentication failed for user
uniqueIdentifier=904042111400,ou=int0,ou=internal,ou=people,o=my
company
.com because of the following exception
[10/31/04 5:31:17:415 EST] 1c1aa27c LTPAServerObj E SECJ0369E:
Authentication failed when using LTPA. The exception is
server2.mycompany.com:389.
[10/31/04 5:31:17:416 EST] 1c1aa27c JaasLoginHelp E SECJ4001E:
Login
failed for
uniqueIdentifier=XXXXX,ou=int0,ou=internal,ou=people,o=my
compnay
.com/server1.mycompany.com:389
[10/31/04 5:31:17:418 EST] 1c1aa27c ContextManage E SECJ0270E:
Failed to
get actual credentials. The exception is .
[10/31/04 5:31:17:421 EST] 1c66a27c LdapRegistryI E SECJ0352E:
Could not
get the users matching the pattern
This is seen in the javacore:
============================
Deadlock detected !!!
--------------------
Thread "Alarm : 153" (0x545D8E30)
is waiting for:
sys_mon_t:0x449F2E48 infl_mon_t: 0x00000000:
byte[][0]:
which is owned by:
Thread "Thread-19" (0x45D70150)
which is waiting for:
sys_mon_t:0x45D7B728 infl_mon_t: 0x45D78148:
com.ibm.ws.security.auth.ContextManagerImpl@313C7ED0/313C7ED8:
which is owned by:
Thread "Alarm : 153" (0x545D8E30)
----------------------------------------------------------------
------
"Thread-19" (TID:0x30FF5070, sys_thread_t:0x45D70150, state:MW,
native
ID:0x356A) prio=5
at
com.ibm.ws.security.auth.ContextManagerImpl.getServerCredential(
ContextM
anagerImpl.java(Compiled Code))
at
com.ibm.ISecurityLocalObjectBaseL13Impl.VaultImpl.getServerCred(
VaultImp
l.java(Compiled Code))
at
com.ibm.ISecurityLocalObjectBaseL13Impl.CredentialsImpl.is_valid
(Credent
ialsImpl.java(Compiled Code))
at
com.ibm.ISecurityLocalObjectBaseL13Impl.CredentialsImpl.get_cred
ential_t
oken(CredentialsImpl.java(Compiled Code))
at
com.ibm.ISecurityLocalObjectTokenBaseImpl.CredentialsImpl.get_cr
edential
_token(CredentialsImpl.java(Compiled Code))
at
com.ibm.ws.security.auth.WSCredentialImpl.getCredentialToken(WSC
redentia
lImpl.java(Compiled Code))
at
com.ibm.ISecurityLocalObjectTokenBaseImpl.WSSecurityContextLTPAI
mpl.init
SecContext(WSSecurityContextLTPAImpl.java(Compiled Code))
at
com.ibm.ws.management.connector.soap.SOAPConnectorClient.process
Internal
(SOAPConnectorClient.java(Compiled Code))
at
com.ibm.ws.management.connector.soap.SOAPConnectorClient.getSecu
rityHead
er(SOAPConnectorClient.java(Inlined Compiled Code))
at
com.ibm.ws.management.connector.soap.SOAPConnectorClient.invokeT
emplate(
SOAPConnectorClient.java(Compiled Code))
at
com.ibm.ws.management.connector.soap.SOAPConnectorClient.invoke(
SOAPConn
ectorClient.java(Compiled Code))
at
com.ibm.ws.management.connector.soap.SOAPConnectorClient.invoke(
SOAPConn
ectorClient.java(Compiled Code))
at $Proxy0.invoke(Unknown Source)(Compiled Code)
at
com.ibm.ws.management.AdminClientImpl.invoke(AdminClientImpl.jav
a(Inline
d Compiled Code))
at
com.ibm.ws.management.AdminServiceImpl.invoke(AdminServiceImpl.j
ava(Comp
iled Code))
at
com.ibm.ws.management.nodeagent.PidWaiter.contact(PidWaiter.java
(Compile
d Code))
at
com.ibm.ws.management.nodeagent.PidWaiter.run(PidWaiter.java:171
)
at java.lang.Thread.run(Thread.java:513)
Native Stack
------------
at 0xD004A384 in _global_lock_common
at 0xD0048D74 in pthread_mutex_lock
at 0xD149F360 in cooperative_mutex_lock
at 0xD149EF54 in sysMonitorEnter
at 0xD137B7D4 in lkMonitorEnter
at 0xD14E515C in _jit_monitorEnterQuicker
at 0xD14EAB38 in JITSigSegvHandler
Local fix Problem summary
****************************************************************
* USERS AFFECTED: WebSphere Application Server users who have *
* enabled security *
****************************************************************
* PROBLEM DESCRIPTION: Deployment Managers, Node Agents *
* or Application Servers may hang. *
****************************************************************
* RECOMMENDATION: *
****************************************************************
Deployment Managers, Node Agents or Application Servers may
hang. This is caused by a deadlock condition in aquiring
a server credential reference.
Problem conclusion
A race condition was removed that may occur while getting
server subject by changing synchronization code.
This APAR is projected to be included in 5.0.2.10 and 5.1.1.3.
Temporary fix
provided test fix
Comments
APAR information |
APAR number |
PQ96894 |
Reported component name |
WAS NETWRK DEPL |
Reported component ID |
5630A3601 |
Reported release |
00A |
Status |
CLOSED PER |
PE |
NoPE |
HIPER |
NoHIPER |
Special Attention |
NoSpecatt |
Submitted date |
2004-11-08 |
Closed date |
2004-11-18 |
Last modified date |
2004-11-18 |
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
Modules/Macros
Publications Referenced
Applicable component levels |
R00A PSY |
UP |
R00H PSY |
UP |
R00I PSY |
UP |
R00P PSY |
UP |
R00S PSY |
UP |
R00W PSY |
UP |
R10A PSY |
UP |
R10H PSY |
UP |
R10I PSY |
UP |
R10P PSY |
UP |
R10S PSY |
UP |
R10W PSY |
UP |
|