WebSphere Application Server V5.0 or V5.1 application server fails to start or becomes unstable after removal from cell due to NoClassDefFoundError
 Technote (troubleshooting)
 
Problem(Abstract)
The IBM® WebSphere® Application Server V5.0 or V5.1 application server on a base node, which has just been removed from a cell, could fail to start or show multiple errors after it has started. These errors are attributed to the Administrative Console application. Steps can be taken to correct this problem and prevent it from happening in the future.
 
Cause
The Administrative Console is not updated by a Fix Pack installation on a federated base node. This is normal and acceptable for a node which is always federated in production. But, this causes side-effects if the node is eventually removed from the cell.
Removing a node from a cell restores the node configuration from the "config\backup" folder. The backup most likely contains the Administrative Console application. If a Fix Pack had been applied to the node when the node was federated, that backup Administrative Console application would contain backlevel code.

In other words, a federated node contains a backup copy of Administrative Console code. If the node was updated with maintenance while it was federated, that backup copy of the code is no longer in synchronization with the rest of the code installed on that node. When the node is removed from the cell, the old, outdated Administrative Console is restored from backup. This is by design, as the updateInstaller should not update information contained in the node configuration backup folder.

If the administrative console becomes corrupted in this manner after removing the node from a cell, the server on that node will not start. Typically, it produces this error in the SystemOut.log:

SRVE0026E: [Servlet Error]-[com/ibm/etools/emf/resource/Context]: java.lang.NoClassDefFoundError: com/ibm/etools/emf/resource/Context
at com.ibm.ws.console.core.User.valueBound(User.java:75)
at com.ibm.ws.webcontainer.httpsession.SessionData.putValueGuts(SessionData.java:719)
at com.ibm.ws.webcontainer.httpsession.SessionData.putValue(SessionData.java:1122) [stack trace truncated for brevity]
 
Resolving the problem
Preventative measure
When planning a cell, consider applying the Fix Pack (and cumulative fixes) before federating the node. That way, the Administrative Console application deployed on that stand-alone node is updated. If the node is federated, the updated Administrative Console code is backed up with that node. If the node is eventually removed from the cell, it will not experience this problem.

This preventative measure is not helpful in situations where nodes have already been federated into a cell, or in situations where a new Fix Pack is published after the nodes have been federated. The two solutions outlined below correct this situation.

WebSphere Application Server Fix Packs, cumulative fixes, and Interim Fixes are collectively referred to as "maintenance" in the solutions outlined below.


Recommended solution
Remove all maintenance applied to this stand-alone node, then re-apply the desired maintenance.

The purpose for re-applying the maintenance is to update the outdated Administrative Console component with code at the proper maintenance level. Removing all the maintenance applied to WebSphere Application Server before re-applying the maintenance is the safest way to ensure that all components are updated to the same maintenance level.

Removing maintenance will have a side effect of altering the setupCmdLine.sh file. That file may revert to a state representing the federated node, which is undesirable for a stand-alone node. Before removing maintenance, take note of the WAS_CELL and WAS_NODE values in that file. After the maintenance is removed, examine the setupCmdLine file again, and ensure that the WAS_CELL and WAS_NODE values are still correct. This helps avoid Error 196 when installing the Fix Pack again later.

This procedure outlines the recommended solution:
  1. Download the latest copy of the updateInstaller utility from the support site:

    updateInstaller for WebSphere Application Server 5.0
    updateInstaller for WebSphere Application Server 5.1

  2. Extract the downloaded updateInstaller file. If the updateInstaller already exists on the system, it can be overwritten with this new updateInstaller.

  3. Remove the node from the cell.

  4. Examine Install_Root /bin/setupCmdLine.sh (file extension is ".bat" on Windows®). Take note of the WAS_CELL and WAS_NODE values.

  5. Remove all Interim Fixes, cumulative fixes, and Fix Packs which have been applied to WebSphere Application Server.

  6. Examine Install_Root /bin/setupCmdLine.sh (file extension is ".bat" on Windows). Make sure that the WAS_CELL and WAS_NODE values are the same as they were before. If the values have changed, edit setupCmdLine and change them back to their previous values.

  7. Install the desired Fix Packs, cumulative fixes, and Interim Fixes again. The Administrative Console component should be updated properly since the node has been removed from the cell.

The Administrative Console should not throw errors when started.

Alternative solution

This solution can be used instead of the primary solution documented above. However, the primary solution is the preferred, recommended solution, since this solution requires a separate, non-federated node which is at exactly the same maintenance level. If there are any doubts as to the status of the separate WebSphere Application Server installation, then do not use this alternate solution.

This solution requires a second installation of WebSphere Application Server, regardless of whether or not it is located on the same physical system as the targeted installation. The second WebSphere Application Server installation is referred to as a stand-alone node. The WebSphere Application Server which requires repair is referred to as the broken node.

The stand-alone node must be at exactly the same maintenance level as the broken node. For example, if the broken node is at version 5.0.2.3 (Fix Pack 2 with Cumulative Fix 3) and Interim Fix PQ99999, then the stand-alone node must also have version 5.0.2.3 with Interim Fix PQ99999. The stand-alone node must not have been federated when maintenance was applied to it. Preferably, the stand-alone node should never have been federated at all. If there is any doubt as to the federation history of the stand-alone node, then do not use this alternate solution; instead, use the primary solution.

If the conditions above are met, then follow this procedure:
  1. Copy the adminconsole.ear file from the $Install_Root /installableApps directory of the stand-alone node to the same directory on the broken node.

  2. Redeploy the Administrative Console on the broken node. For instructions on how to redeploy an Administrative Console, refer to this technote:

    http://www.ibm.com/support/docview.wss?&context=SSEQTP&uid=swg21163938
 
 
Cross Reference information
Segment Product Component Platform Version Edition
Application Servers Runtimes for Java Technology Java SDK
 
 


Document Information


Product categories: Software > Application Servers > Distributed Application & Web Servers > WebSphere Application Server > Install
Operating system(s): Windows
Software version: 5.1
Software edition:
Reference #: 1177359
IBM Group: Software Group
Modified date: Aug 31, 2007