APAR status
Closed as program error.
Error description
Exception in nodeagent SystemOut.log file.
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.next(HashMap.java:731)
at com.ibm.ws.management.event.DownstreamServerManager.
updateServerListeners(DownstreamServerManager.java:223)
at
com.ibm.ws.management.event.DownstreamServerManager.setFilter
(DownstreamServerManager.java:201)
at com.ibm.ws.management.event.ClientNotificationService.
addClientListenerInternal(ClientNotificationService.java:132)
at com.ibm.ws.management.event.ClientNotificationService.
addNotificationListenerExtended(ClientNotificationService.java:1
12)
at com.ibm.ws.management.AdminServiceImpl.
addNotificationListenerExtended(AdminServiceImpl.java:847)
at com.ibm.ws.management.nodeagent.PidWaiter.
listenForJ2EEStateStoppingNotifications(PidWaiter.java:108)
at
com.ibm.ws.management.nodeagent.PidWaiter.<init>(PidWaiter.java:
93)
at com.ibm.ws.management.nodeagent.NodeAgent.monitorProcess
(NodeAgent.java:951)
at com.ibm.ws.management.nodeagent.NodeAgent.handleNotification
(NodeAgent.java:862)
at com.ibm.ws.management.event.ListenerInfo.handleNotification
(ListenerInfo.java:67)
at com.ibm.ws.management.event.
NotificationDispatcher$DispatchANotificationToAListener.run
(NotificationDispatcher.java:334)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:672)
Exception shows a concurrency issue in notification. More
specifically, as each
running application server is discovered by the node agent, it
is added to a list of downstream servers maintained by
notification logic. A
ConcurrentModificationException would occur when multiple thread
tries to update this list at the same time. This problem
probably is not
always reproducible as it is dependent on the timing. In
earlier releases of 5.0.2.x, where it took more time to process
notifications,
this problem would not likely to happen. In the later 5.0.2.x
releases, where notifications are processed faster, this problem
is more likely to show up.
-
When this exception is seen there is a delay stopping
applications. When the stop is issued from the admin
console there will be a 3 minute delay between when the
first app server stops the application and when the
next app server stops it.
Local fix
Not Available
Problem summary
****************************************************************
* USERS AFFECTED: Websphere application server version 5.0.2 *
* and 5.1.X users who try to stop and start *
* the clusters and individual cluster members. *
* Users also face the problem when they try *
* to stop application installed on clusters. *
* Users might also face this problem also *
* when there are concurrent administrators. *
****************************************************************
* PROBLEM DESCRIPTION: 1.ConcurrentModificationException in *
* nodeagent systemout.log *
* 2.stop application(j2ee.state.stopped) *
* notification is missed because of non *
* availability of the listener. *
****************************************************************
* RECOMMENDATION: *
****************************************************************
Customers sees ConcurrentModification exception in their
SystemOut.log.
1.Problem: Client upgraded system from 5.0.2.2 to 5.0.2.9 and
now they see the nodeagents throw the following error due to
concurrent administrators
Customer sees the delay in there j2ee.stop.notification.while
trying to stop the applications. 2. this is nodeA running
dmgr,na and two cluster members, nodeB running na
and two more cluster members. Stopping an eApp running in the
members will cause a considerable delay (about 6 min) with one
eApp stopped in mem1, then mem2, then mem3 and so on.
Problem conclusion
1.Synchronized the block which was causing the error.
The fix will be included in 5.0.2.11 and 5.1.1.5
Please refer to the recommended updates page for delivery
information:
http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980
Temporary fix Comments
APAR information |
APAR number |
PQ99966 |
Reported component name |
WAS NETWRK DEPL |
Reported component ID |
5630A3601 |
Reported release |
00S |
Status |
CLOSED PER |
PE |
NoPE |
HIPER |
NoHIPER |
Special Attention |
NoSpecatt |
Submitted date |
2005-01-25 |
Closed date |
2005-03-24 |
Last modified date |
2005-04-13 |
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
PK04172
Modules/Macros
Publications Referenced
|
Fix information |
Fixed component name |
WAS NETWRK DEPL |
Fixed component ID |
5630A3601 |
Applicable component levels |
R003 PSY |
UP |
R00A PSY |
UP |
R00H PSY |
UP |
R00I PSY |
UP |
R00P PSY |
UP |
R00S PSY |
UP |
R00W PSY |
UP |
R103 PSY |
UP |
R10A PSY |
UP |
R10H PSY |
UP |
R10I PSY |
UP |
R10P PSY |
UP |
R10S PSY |
UP |
R10W PSY |
UP |
|