Error messages similar to the ones listed at the top of this article may occur when some base operating system libraries are:
1. Installed at older levels and need to be upgraded
2. Not at a consistent level with other base operating system libraries
3. Not correctly loaded into the system kernel
The above conditions may be the result of one or more bos.* filesets (e.g., bos.perf.perfstat, bos..perf.libperfstat, bos.mp64, bos.rte.libc, bos.rte.control, bos.rte.compare)
1. Not havng been installed at the correct or current levels
2. Having become inconsistent due to the manner in which AIX updates have been applied
3. Having become inconsistent due to an incomplete installation or upgrade
4. Having become inconsistent due to a failed installation or upgrade
Please follow all of the instructions listed below to resolve this known issue.
When contacting IBM Support, the support specialists will request confirmation that all of these steps have been completed and the inconsistences have been corrected.
The first step in resolving these messages is to work with the system administrator(s) to identify and resolve all mainteance level inconsistencies by performing the following steps:
1. Execute the following commands, while logged in as the root user, from a command prompt:
# oslevel -rl "$( oslevel -rq | sort -n | tail -1 )" 2>&1 | tee /tmp/oslevel-rl.out
2. Resolve all inconsistencies reported in the file /tmp/oslevel-rl.out file.
3. Repeat steps A.1 and A.2 until all inconsistencies have been resolved and no inconsistences are reported in the /tmp/oslevel-rl.out file.
4. Start the Java application (or run the command java -version). If the messages continue to be displayed, proceed to Step B. Resovle Service Level Inconsistencies.
The next step is to continue to work with the system administrator(s) to identify and resolve all service level inconsistencies by performing the following steps:
1. Execute the following commands, while logged in as the root user, from a command prompt:
# oslevel -sl "$( oslevel -qs | sort -n | tail -1 )" 2>&1 | tee /tmp/oslevel-sl.out
2. Resolve all inconsistencies reported in the file /tmp/oslevel-sl.out file.
3. Repeat steps B.1 and B.2 until all inconsistencies have been resolved and no inconsistences are reported in the /tmp/oslevel-sl.out file.
4. Start the Java application (or run the command java -version). If the messages continue to be displayed, proceed to Step C. Resovle lppchk Inconsistencies.
The next step is to continue to work with the system administrator(s) to identify and resolve all lppchk inconsistencies by performing the following steps:
1. Execute the following commands, while logged in as the root user, from a command prompt:
# lppchk -v 2>&1 | tee /tmp/lppchk-v.out
2. Resolve all inconsistencies reported in the file /tmp/lppchk-v.out file.
3. Repeat steps C.1 and C.2 until all inconsistencies have been resolved and no inconsistences are reported in the /tmp/lppchk-v.out file.
4. Start the Java application (or run the command java -version). If the messages continue to be displayed, proceed to the next step (step C.5).
5. Execute the following commands, while logged in as the root user, from a command prompt:
{Warning: this command performs an extensive file verification check and will take several minutes to complete}
# lppchk -c 2>&1 | tee /tmp/lppchk-c.out
6. Resolve all inconsistencies reported in the file /tmp/lppchk-c.out file.
7. Repeat steps C.5 and C.6 until all inconsistencies have been resolved and no inconsistences are reported in the /tmp/lppchk-c.out file.
8. Start the Java application (or run the command java -version). If the messages continue to be displayed, proceed to Step D. Reboot the LPAR.
The next step is to continue to work with the system administrator(s) to identify and resolve bos.* fileset inconsistencies by performing the following steps:
1. Execute the following commands, while logged in as the root user, from a command prompt:
# lslpp -l "bos.mp64" "bos.rte.*" "bos.perf*" 2>&1 | tee /tmp/lspp-l-bos.out
2. Verify that the highest level for all of the filesets is the same for both the Path: /usr/lib/objrepos and Path: /etc/objrepos sections. If the highest levels are not consistent, then take the appropriate actions to ensure the levels are consistent.
3. Start the Java application (or run the command java -version). If the messages continue to be displayed, proceed to the next step (step D.4).
4. Once it has been confirmed that the highest installed level is consistance and the messages continue to be displayed, download and install the most current level for all of the bos.perf.* filesets (especially the bos.perf.perfstat and bos.perf.libperfstat)
5. Start the Java application (or run the command java -version). If the messages continue to be displayed, proceed to the Step E. Reboot the LPAR.
In rare situations, customers have reported that rebooting the LPAR had resolved the situation. If after rebooting the LPAR the messages no longer appear, that is an indication that the state of some system libraries became inconsistent in the kernel. The reboot re-initialiizes the state of these libraries. This should be a one time operation.
If, after completing all of the above steps, the messages continue to occur, please follow these actions:
1. Contact IBM to open a new IBM Support Call
a. If assistance is required to resolve the inconsistencies reported by oslevel and lppchk, request for the call to be routed to the AIX Install Team.
b. If there are no inconsistencies reported by oslevel and lppchk and the issue continues, request for the call to be routed to the AIX Kernel Team.
2. When opening the support call, confirm that you have completed all steps in this document.
3. Collect and package the following data to your IBM Support Call (a.k.a. PMR)
a. The /tmp files created in the above steps (i.e., /tmp/oslevel-rl.out, /tmp/oslevel-sl.out, /tmp/lppchk-v.out, /tmp/lppchk-c.out)
b. An AIX system snap (e.g., the command "snap -ac")
4. Upload the packaged data
Using one of the IBM secure upload methods listed on the "IBM Java for AIX MustGather: How to upload diagnostic data and testcases to IBM" web page, upload the packaged data:
http://www-01.ibm.com/support/docview.wss?uid=isg3T1022619
Document Type: | Technical Document |
Content Type: | General |
Hardware: | all Power |
Operating System: | all AIX Versions |
IBM Java: | all Java Versions |
Author(s): | Roger Leuckie |
Reviewer(s): | Barkat Mohammad |