WebSphere WebSphere Application Server Express, Version 6.0.x Operating Systems: AIX, HP-UX, Linux, Solaris, Windows

Tips for troubleshooting service integration messaging

This topic provides a set of specific tips to help you troubleshoot problems with service integration messaging.

Problem determination for a data store

You can perform a dump, in reduced form, of the data in the data store for a messaging engine. The output is intended for use by IBM Service personnel.

If there is a problem with the data in the data store, it can be hard to diagnose from the trace output. However, you can create a dump, in XML format, of the data in the data store. This makes diagnosis easier because it is a human readable representation that can be transformed to other formats as required.

The dump is created as an XML file in the $WAS_HOME/logs/server1 directory. The file is named according to the format: messaging_engine_nameUUIDtimestamp.xml

The format of the file is illustrated in the following example:
<MessageStore>
    <itemStreams>
        <ItemStreamLink id="0" state="Available">
            <class>com.ibm.ws.sib.msgstore.ItemStream</class>
            <priority>5</priority>
            <canExpireSilently></canExpireSilently>
            <storageStrategy>STORE_NEVER</storageStrategy>
            <expiryTime>0</expiryTime>
            <sequence>0</sequence>
            <tranID>null</tranID>
            <tickValue>0</tickValue>
            <items>
                <ItemLink id="2" state="Available" refCount="3" refCountDecreasing="false">
                    <class>com.ibm.ws.sib.msgstore.Item</class>
                    <priority>5</priority>
                    <canExpireSilently></canExpireSilently>
                    <storageStrategy>STORE_NEVER</storageStrategy>
                    <expiryTime>0</expiryTime>
                    <sequence>1</sequence>
                    <tranID>null</tranID>
                    <tickValue>0</tickValue>
                </ItemLink></items></ItemStreamLink></itemStreams></MessageStore>

The XAResourceNotAvailableException exception

When the deleteNode command is used for a node that hosts messaging engines, those messaging engines are deleted. When new messaging engines are re-created following the addNode command, they have different identifiers and so during transaction recovery it is not possible to connect to the old messaging engines. A message identifying the XAResourceNotAvailableException exception is generated in the SystemOut.log file for each server that hosts a messaging engine.

To solve this problem, you must follow the procedure described in Resolving in-doubt transactions.

Problems when recreating a service integration bus

If you delete a service integration bus, and later create a new bus with the same name, the messaging engine fails to start and messages like the following are generated in SystemOut.log:
[8/11/04 21:55:01:439 CDT] 0000000f SibMessage    I   
[LateBus:xyzsun15.server1-LateBus] isAlive: MessagingEngine suffered common mode error. Correct error (see logs) and restart server.
[8/11/04 21:55:01:468 CDT] 0000000f SibMessage    I   
[LateBus:xyzsun15.server1-LateBus] isAlive: MessagingEngine will be stopped because of common mode error. No failover will occur.
[8/11/04 21:55:01:493 CDT] 0000000f SibMessage    I   
[LateBus:xyzsun15.server1-LateBus] Messaging Engine 
xyzsun15.server1-LateBus not in state from which stop is valid: Starting
[8/11/04 21:55:01:513 CDT] 0000000f SibMessage    I   
[LateBus:xyzsun15.server1-LateBus] isAlive: MessagingEngine stopped because of common mode error. Correct error (see logs) and restart server.
[8/11/04 21:57:01:431 CDT] 0000000e SibMessage    I   
[LateBus:xyzsun15.server1-LateBus] isAlive: MessagingEngine suffered common mode error. Correct error (see logs) and restart server.

The reason is that the database directory for the messaging engine still exists after deletion of the bus and must be manually removed. To delete the Cloudscape database for a non-existent messaging engine, you must delete the database directory that is located in profile_root/databases/com.ibm.ws.sib, where profile_root is the directory in which profile-specific information is stored.

You must stop WebSphere Application Server before you can delete the database files.

For other databases, you can either delete all of the rows from the data store tables, or you can drop all of the tables. The names of the data store tables all begin with SIB, and are in the schema that you configured for the data store.

For more information , see Data store life cycle.

Problems when re-creating bus members

If you previously created a bus and added a bus member using the DEFAULT data source, when you attempt to recreate this bus member and you therefore delete the bus member and then try to add it again, the following exception appears:
ADMG0037E: A new instance of the DataSource object  cannot be created because the jndiName attribute of an existing DataSource object has the same value as jdbc/com.ibm.ws.sib/<NODENAME>.<SERVERNAME>-<BUSNAME>
Note: The bus member is recreated but the messaging engine fails to start.
To resolve the problem, delete the data source manually. To do this, use the administrative console to complete the following steps:
  1. Delete the bus member that is causing the exception.
  2. Select Resources > JDBC Providers.
  3. Change the scope to the scope of the data source that you want to delete. For example, on a single server install, you would choose Server, but this may vary with other topologies.
  4. In the list of data sources, click Cloudscape JDBC Provider. Note: Select the non XA option.
  5. On the Configuration tab, under Additional Properties, click Data sources.
  6. Select the check box next to the data source specified in the error message.
  7. Click Delete.
  8. Save your changes to the master configuration.
You should now be able to recreate the bus member without difficulty.

Problems when attempting to communicate with a renamed foreign bus

The administrative console panel used for configuring the properties of a service integration bus link, also allows you to change the foreign bus name that the link is pointing to. However, the foreign bus name must not be altered once it has been configured. If it is, any messaging engines that already hold state information about the link will not be able to use the link unless the foreign bus name is reset to its previous value.

The JMSException with a wrapped SILimitExceeded exception

When the number of messages held by a destination reaches its limiting threshold, any attempt to send a message to that destination fails with a JMSException with a wrapped SILimitExceeded exception. The destination continues to fail with this exception until the number of messages held by the destination reduces below the limiting threshold.

To obtain an accurate count of the number of available message, you can monitor the Available Message Count PMI statistic for queue and topicspace destinations. If the number of available messages increases, take action to balance the system. Consider stopping producers from sending new messages until the destination consumes the available messages.

Examine the following list for possible causes and solutions for this problem:

Corruption errors on restart

It is possible, although rare, for a messaging engine, destination or link to be corrupted after a restart of the system. If this occurs you will see a message indicating the problem. If the problem lies with the messaging engine, the messaging engine will not start. If a destination or link is corrupted, the relevant messaging engine will start, but the destination or link will not be usable on that messaging engine.

If you do not know the cause of the problem, contact your IBM service representative to establish the cause before attempting to resolve the situation.

If you know the cause of the problem, for example, you are aware of an issue with your database, resolve it by completing the following steps:
  1. Ensure that the WCCM configuration files are synchronized across your system by clicking System administration > Nodes > Full resynchronize. This operation can take several minutes to run.
  2. If the problem still exists, perform one of the following tasks:

Use this task to enable you to retrieve the status of messaging engines in the administrative console

To be able to retrieve the status of messaging engines, you must be logged into the administrative console with at least monitor authority. If you do not have this authority, the messaging engine status is displayed as "Unavailable", even if the messaging engine has started.

If you are not logged in with the authority needed to retrieve the status of messaging engines, an error message like the following is logged in the server's systemOut log file:
[4/20/05 10:49:57:083 CDT] 0000004b RoleBasedAuth A   SECJ0305I: The role-based  authorization check failed for admin-authz operation   SIBMessagingEngine:stateExtended.  The user UNAUTHENTICATED (unique ID:  unauthenticated) was not granted any of the following required roles:  administrator, operator, configurator, monitor.
Where the user ID actually shown in the message is the user ID that you used to log in to the administrative console.
Related concepts
Data store

Reference topic

Terms of Use | Feedback

Last updated: 2 Aug 2005
http://publib.boulder.ibm.com/infocenter/ws60help/index.jsp?topic=/com.ibm.websphere.pmc.express.doc\ref\rjk_prob0.html

© Copyright IBM Corporation 2004, 2005. All Rights Reserved.
This information center is powered by Eclipse technology. (http://www.eclipse.org)