|
Problem(Abstract) |
This article describes issues specific to the "ODM"
experienced while installing and uninstalling WebSphere® Application
Server V5.0, 5.1 and 6.0 as well as IBM® HTTP Server V1.3, 2.0, and 6.0 on
AIX®. Also, solutions for common issues are discussed. |
|
|
|
Cause |
This document provides a quick overview for identifying
and recovering from issues related to the AIX ODM VPD. Other in-depth
articles are available on related subjects. For more information, see
Explanation
of AIX ODM VPD and its relationship with WebSphere Application Server
products.
Under some circumstances, the information associated with WebSphere
Application Server or IBM HTTP Server product installations stored in the
AIX ODM VPD can become corrupt or partially deleted. This will prevent the
WebSphere Application Server or IBM HTTP Server product installers from
working properly. The product installers cannot automatically recover from
these situations; therefore, it is necessary to follow a manual cleanup
process to recover.
The "AIX ODM VPD" is AIX's repository of data associated with software
installations on the system. The ODM VPD is the "Object Data Manager -
Vital Product Data", which (among other things) provides functionality
similar to the Windows® Registry or the RPM database on Linux-based
platforms. For the purpose of brevity in this article, the AIX ODM VPD
will simply be referenced as the ODM.
Products covered by this document
This article applies to both WebSphere Application Server and IBM HTTP
Server products. The methods described here apply to the following
products and releases:
- WebSphere Application Server V5.0, 5.1, and 6.0
- WebSphere Application Server, Enterprise Edition V5.0
- WebSphere Business Integration Server Foundation V5.1
- IBM HTTP Server V1.3, 2.0, and 6.0
The document is applicable to any maintenance release of these product
versions. For the purpose of brevity, this article will refer to the
products collectively as "WebSphere Application Server products".
Causes for ODM issues
Here are the most common reasons for experiencing ODM issues with
WebSphere products:
- The WebSphere Application Server product was deleted from
the system without running the product's uninstaller program, or the
uninstaller program failed to properly clean up the system.
- WebSphere Business Integration Server Foundation (and
Enterprise Edition) use separate uninstaller programs from the underlying
products which they extend. If the underlying product (such as a "base"
application server) is uninstalled before WebSphere Business Integration
Server Foundation (or Enterprise Edition), then it might become impossible
to properly uninstall WebSphere Business Integration Server Foundation (or
Enterprise Edition).
- The WebSphere Application Server products covered in
this document must be installed as the root user. The products must also
be uninstalled as the root user. If the product is uninstalled under an
underprivileged user's authority, the uninstaller might be unable to
modify the ODM.
- The AIX system was migrated from an earlier version to
version 5.1, 5.2, or 5.3. There is a known issue with the migration
software in those releases of AIX which drops an important database table
in the ODM. This adversely affects the WebSphere Application Server
product installers. There is no reliable method to avoid this unless
precautions are taken before and after the migration process. If this is
discovered after the operating system migration takes place, then it is
necessary to clean up the ODM.
- An installation fails before completion. When the
installer fails, it might not have an opportunity to write the uninstaller
utility to the file system. As a result, there is no automated method
available to recover.
Symptoms which indicate ODM issues
There are several factors which indicate that the WebSphere Application
Server product installer is experiencing difficulty with the ODM. For
hints about where to find the product's installation log files, see the
table below.
- The installer fails or hangs in the installation wizard,
or it claims that it successfully installed the product nearly
instantaneously. In either case, the presence of this stack trace error in
the installation log is a major indicator of an ODM-related installation
issue.
STACK_TRACE: 11
java.lang.NullPointerException at
com.installshield.product.SoftwareObjectUtils.updateKey(SoftwareObjectUtils.java:21)
at
com.installshield.product.service.product.PureJavaProductServiceImpl$Inst
allProduct.addToVPD(PureJavaProductServiceImpl.java:4664)
at
com.installshield.product.service.product.PureJavaProductServiceImpl$Inst
allProduct.checkInstalledVersion(PureJavaProductServiceImpl.java:4632) |
|
The highlighted methods (updateKey and
addToVPD) are the key indicators. |
|
- The installer claims that the product installation was
successful, but the wizard hangs at the end of the process. The
installation process proceeds much faster than it should have, perhaps
taking mere seconds to claim that the product is installed. Or, perhaps
the installation appears to proceed normally, but attempts to start the
server fail.
In this case, the product installation log file contains several
occurrences of this phrase:
Did not replace installed
object (name) with object (name) |
|
This typically means that the installer detected data
related to a previous installation instance of the product. As a result,
the installer will skip the installation of critical components. |
|
- If the AIX system has been recently migrated to a newer
release (for example, migrating from AIX 5.1 to AIX 5.2), then a critical
ODM table is dropped during the process. It is possible to test whether or
not the ODM table has been dropped by issuing the following command:
ODMDIR=/usr/lib/objrepos odmget vendor
>vendor.txt |
|
Examine the resulting vendor.txt file. If it is
blank, then the vendor table is empty. If a WebSphere Application Server
product is currently installed on this system and the vendor table is
blank, then it is likely that the vendor table was dropped; therefore, the
ODM needs to be cleaned up. |
|
Install Log Locations
Product |
Location |
WebSphere Application Server V5.0, 5.1, and 6.0 |
/tmp/log.txt
<install_root>/logs/log.txt |
WebSphere Enterprise Edition V5.0 |
/tmp/WAS.PME.install.log
<install_root>/logs/WAS.PME.install.log |
WebSphere Business Integration Server Foundation V5.1 |
/tmp/WAS.WBISF.install.log
<install_root>/logs/WAS.WBISF.install.log |
IBM HTTP Server V1.3, 2.0, and 6.0 |
/tmp/log.txt
<install_root>/log.txt |
|
|
|
Resolving the
problem |
The following is an overview of the recovery process:
The recovery process can be handled in two different ways. The admin can
choose to do a "complete clean", also known as a "wipe", and remove all
entries from the ODM related to WebSphere. Alternatively, the admin may
choose to do a "targeted clean", which will clean select entries related
to a single WebSphere product installation.
In general, IBM support recommends the "complete clean" procedure. Due to
the nature of ODM corruption, it is sometimes impossible to determine
which WebSphere product entries need to be removed or preserved during a
"targeted clean". Therefore, the "complete clean" is a more reliable
method of recovery. Also, the disadvantages associated with the "complete
clean" are limited, especially considering that a "complete clean" does
not affect the ability of an installed WebSphere product to continue to
run.
Choose the cleanup method (Complete or Targeted)
Read the following scenarios and choose the first scenario which most
closely matches the current situation. The proposed solution would be the
best course of action.
- The AIX system was migrated from an earlier version to
version 5.1, 5.2, or 5.3, and WebSphere Application Server products
were installed on the system at the time of the migration. The migration
process drops a necessary ODM table, thereby corrupting the ODM. Without
exception, it is necessary to use the complete cleanup procedure to
recover.
- There is only one installation of an WebSphere
Application Server product on the system, and it failed to
properly install, or there was a problem during the process of
uninstalling it. The best solution is to use the complete cleanup procedure to
recover.
- There are multiple copies of WebSphere Application
Server on the system, and the newest product instances failed to
properly install, or there was a problem during the process of
uninstalling them. The older instances of WebSphere Application Server are
established and configured, and there are no plans to install
additional products on top of those older instances (such as Lotus Portal
Server, DB2 Content Manager, or WebSphere Commerce Suite). The older
instances will only be altered by typical maintenance pack upgrades. The
best solution is to use the complete cleanup procedure to
recover.
- Similar to the preceding scenario, there are multiple
copies of WebSphere Application Server on the system, and one or more
of them failed to properly install or uninstall. Unlike the previous
scenario, the existing, successfully installed instances of WebSphere
Application Server are not fully established. Products which take
advantage of WebSphere Application Server may be installed at some point
in the future (such as Lotus Portal Server, DB2 Content Manager, or
WebSphere Commerce Suite). The best solution is to use the targeted cleanup procedure.
(It is also possible to use the complete cleanup procedure then reinstall
the needed WebSphere Application Server products.)
Information about the complete cleanup procedure
The complete cleanup
procedure is recommended for most cases when cleaning up the WebSphere
entries in the AIX ODM. The "complete clean" will search for all entries
related to WebSphere products and delete them. The set of entries which
need to be cleaned are consistent and are not dependant on product
installation directories or the number of instances which are already
installed. As a result, the complete clean script does not require input
or intervention, and immediately cleans the system upon execution.
The complete cleanup procedure has several advantages:
- A complete cleanup helps guarantee that ODM entries from
any previously installed instances of WebSphere products do not interfere
with future installations.
- The complete cleanup procedure uses a one-pass script that
requires no input.
- A complete cleanup is immune to the typical issues
WebSphere encounters with the ODM. Even if some WebSphere-related entries
are missing from the ODM, the complete cleanup will still be able to wipe
out the necessary WebSphere-related entries.
- The complete cleanup procedure cleans ODM entries without
removing the actual product files. The application servers remain intact
and can run without issues.
- The Embedded Messaging feature is preserved. Future
installations of WebSphere Application Server V5.0 or 5.1 can continue to
use the existing Embedded Messaging code.
Administrators utilizing the complete clean procedure should also be aware
of the following:
- As a convenience feature, the V5.0 and 5.1
updateInstaller's wizard mode attempts to detect the installed product
locations using data from the ODM. When the ODM entries are removed, the
automatic detection is disabled. Administrators will need to manually
specify the location of the WebSphere product in order to update it. (The
V5.0 and 5.1 updateInstaller silent mode is unaffected, since it is always
necessary to specify the location for silent mode. Also, this does not
affect the V6.0 updateInstaller.)
- Certain products which install "on top of" an existing
application server (such as Lotus Portal Server or WebSphere Commerce
Suite) will look for WebSphere-related ODM entries as part of the
installation process. Those products will need to be installed on top of a
new installation instance of WebSphere Application Server after the
WebSphere ODM entries are cleaned up.
- The "complete cleanup" removes the vpd.properties file
from /usr/lib/objrepos. In most cases, WebSphere Application
Server is the only product present on a system which utilizes the
vpd.properties file. (If the beginning of each line in the file
contains "IHS" or "WS", then it is safe to assume that this is true.) The
file is backed up in the same directory which contains the cleanup
script.
How to run the complete cleanup procedure
The complete cleanup procedure, with the complete cleanup scripts, is
documented in technote 1198862.
Refer to that technote for further details.
Once the complete cleanup procedure is complete, the ODM is wiped clean
of WebSphere (or IBM HTTP Server) related entries. The existing
application servers will continue to operate as before.
Overview of process to completely remove products from system
The complete cleanup procedure results in a cleanup of ODM entries, but it
does not remove product files from the file system. Completely removing
the products requires further steps to be taken. If the administrator is
interested in completely removing the WebSphere Application Server
product, then follow the procedure below.
- Follow the procedure documented in technote 1198862.
- Delete the directories which contain the WebSphere Application Server
products.
Note: The application server product data is typically contained
in one "core" directory (such as /usr/WebSphere/AppServer).
WebSphere Application Server V6.0 products might also use profile
directories which can exist at any location of the administrator's
choosing (the default is /usr/WebSphere/AppServer/profiles).
- If the WebSphere Application Server V5.0 or 5.1 Embedded Messaging
feature was installed, then invoke the smit utility (or its text
shell counterpart, smitty). Instruct smit to remove all
packages that start with the name mqm.* or wemps.*.
These packages represent the Embedded Messaging component of WebSphere
Application Server V5.0 or 5.1.
Note: Do not remove the mqm.* and wemps.*
packages if it is necessary to preserve the Embedded Messaging feature for
other application servers installed on the same system. The Embedded
Messaging packages are not touched by the "complete clean" script because
the Embedded Messaging products use a different ODM scheme which is
unaffected by problems WebSphere Application Server V5.0 or 5.1 might
encounter with the ODM.
Information about the targeted cleanup procedure
The "targeted clean" will search for entries related to one particular
WebSphere product installation, based on the location where that product
is or was previously installed, and it will evaluate each entry in an
effort to preserve entries shared by other WebSphere product instances.
The set of entries cleaned by this procedure is entirely dependant on the
input provided, so as a precaution, it is treated as a two-stage process.
During the first stage, the script runs queries but does not alter data,
and produces a second script with customized cleanup commands. The
administrator can check the script and choose whether or not to execute
it.
The targeted cleanup script has these advantages:
- The targeted cleanup script cleans conservatively and
attempts to preserve ODM entries related to other products. This is
desirable in cases where it is essential that ODM entries for other
WebSphere related are preserved, such as when an "extension product" (such
as Lotus Portal Server or WebSphere Commerce Suite) will be installed.
- The targeted cleanup scripts are treated as a two-stage
process. In the first stage, the ODM is queried, and no data is altered.
The results of the first stage are recorded into a script, which can be
reviewed at the administrator's convenience. The cleanup will take place
only if the administrator explicitly runs the generated script.
- As with the "complete cleanup" procedure, the targeted
cleanup procedure cleans ODM entries without removing the actual product
files. The application servers remain intact and can run without issues.
- As with the "complete cleanup" procedure, the Embedded
Messaging feature is preserved. Future installations of WebSphere
Application Server V5.0 or 5.1 can continue to use the existing Embedded
Messaging code.
There are some disadvantages associated with the targeted cleanup script:
- The quality of cleaning the targeted cleanup script is
capable of providing can only be as good as the quality of the WebSphere
ODM entries beforehand. If the ODM entries are missing (such as the case
when migrating from an earlier release of AIX to AIX v5.1, v5.2, or v5.3),
then the targeted cleanup script cannot completely clean up a WebSphere
product's entries.
- Depending on the state of the ODM, the targeted cleanup
script may take some time (5 to 30 minutes) to complete.
- The administrator must take separate steps to clean up the
vpd.properties file associated with the ODM.
- If the targeted cleanup scripts are unable to produce the
desired results, then it is necessary to fall back on the complete cleanup
procedure in order to clean up the ODM.
How to run the targeted cleanup procedure
The targeted cleanup procedure, with the associated cleanup scripts, is
documented in technote 1191336.
Refer to that technote for further details.
Note: Be sure to follow the directions up to the last step
documented in that technote. It is important to clean the
vpd.properties file as documented in that technote.
Once the targeted cleanup procedure is complete, the entries associated
with one particular instance of WebSphere (or IBM HTTP Server) are removed
from the ODM. The other products on the system, both the "other" products
and the "targeted" product, will continue to function as before.
Information about the cleanup procedures and Embedded Messaging
Both the "complete clean" and "targeted clean" scripts do NOT remove the
Embedded Messaging feature. This is of great concern to many clients,
because administrators familiar with the Embedded Messaging feature (which
is a subset of MQ product packages) are aware that Embedded Messaging is
disabled if its ODM entries are removed. Neither of the cleanup solutions
presented in this technote will remove the ODM entries associated with
Embedded Messaging. Instances of WebSphere Application Server V5.0 or 5.1
installed after cleanup can continue to take advantage of the existing
Embedded Messaging feature. |
|
|
|
|
Cross Reference information |
Segment |
Product |
Component |
Platform |
Version |
Edition |
Application Servers |
IBM HTTP Server |
Uninstall |
AIX |
6.0, 2.0.47, 1.3.28, 1.3.26 |
Edition Independent |
Application Servers |
Runtimes for Java Technology |
Java SDK |
|
|
|
|
|
|