MustGather: Security problems with WebSphere Application Server

Technote (FAQ)
Problem
MustGather for security problems with WebSphereŽ Application Server. Gathering this information before calling IBM support will help familiarize you with the troubleshooting process and save you time.
Solution
If you have already contacted support, continue on to the component specific MustGather information. Otherwise, click: MustGather: Read first for all WebSphere Application Server products.

Important: Make sure to review the troubleshooting guides at the bottom of this page before calling support.

Security specific MustGather information
  • Required information for all releases of V4.0 and V5:
    1. Identify your environment - client, server platform, and software.

    2. Description of your problem and symptoms, including any exceptions.

    3. Description of the steps taken to reproduce your problem.

    4. The operations being performed when the problem happens, if known.

    5. If the problem is intermittent or consistent.

    6. Identify WebSphere Application Server version (V4 or V5?) and PTFs.

    7. List of fixes applied to WebSphere.

    8. Identify if programmatic logins are being used.

  • Required files for releases of V4.0:
    1. XML export of the administrative console. To get this, open the administrative console and select Console > Export to a file.

    2. All the logs in the install_root/logs directory for the period of time corresponding to the problem: stdout, stderr, tracefile, activity.log, and so on.

    3. The administrative server tracefile from the install_root/logs directory.

    4. The approximate time the problem occurred.

    5. The sas.server.props and sas.client.props files from the install_root/properties directory.

    6. The product.xml file for install_root/config directory.

    7. Follow instructions tosend diagnostic information to IBM support.

    If instructed to do so by support, take a security and SAS trace of the problem in addition to the preceding information:
    1. Detach the seclogger40.jar (attached below) file to the install_root/classes directory.

    2. Run the administrative console, expand the topology frame and highlight the application server which is going to be traced to display its properties. Click on the Services tab, click on the Trace Service in the services table, and then click Edit Properties.

    3. Enter the following in the Trace Specification field:

      SASRas=all=enabled:com.ibm.ejs.security.*=all=enabled

    4. Enter a fully qualified hostname in the Trace Output field.

      For example:
      C:\WebSphere\AppServer\logs\appsecuritytrace

    5. Click OK and then click Apply. Make sure the application server is started.

    6. Make a backup copy of the install_root/bin/admin.config file, edit the original, and add or edit the following two lines:

      com.ibm.ejs.sm.adminServer.traceString=SASRas=
      all=enabled:com.ibm.ejs.security.*=all=enabled

      com.ibm.ejs.sm.adminServer.traceOutput=
      install_root
      /logs/adminsecuritytrace

    7. Stop the application server.

    8. Delete or rename all the logs in install_root/logs directory. This ensures that the logs are fresh.

    9. Start the application server and recreate the problem.

    10. It is very important to make note of the time the problem occurs, the user ID, and the exact URL being invoked.

    11. Collect the information from step 10, the admin security trace, and the application server security trace along with the other required files discussed above.

    12. Follow instructions tosend diagnostic information to IBM support.

  • Required files for releases of V5:
    • For WebSphere Application Server V5.0.2 and higher, the collector JAR file or files corresponding to the time when the problem occurred. To collect this, perform the following steps:
      1. Delete or rename the logs in the install_root/logs directory. If you are running the deployment manager (dmgr), also delete the logs in the dmgr_install_root/logs directory.

      2. Start the deployment manager (if applicable), the node agent (if applicable), and the Application Server.

      3. Recreate the problem. It is very important to note the approximate system time that the problem occurs, the user ID, and the URL being invoked.

      4. Run the install_root/bin/collector.sh or the collector.bat files.

        The collector tool must be run as root user. Do not run this file directly from the /bin directory or any other WebSphere Application Server directory. Do not user the -summary option.

        Follow the instructions for running the collector tool in the WebSphere Application Server Information Centers:
      5. If you are running the Network Deployment (ND) edition, run the dmgr_install_root/bin/collector.sh or collector.bat file using the same instructions as in step 3.

    • For WebSphere Application Server V5.0.0 or V5.0.1 only, the collector tool is available for these versions, but the JAR file produced is extremely large. Alternatively, you can provide the following:
      1. The JVM logs corresponding to the problem, SystemOut.log and SystemErr.log.

      2. All logs under the ffdc directory.

      3. Security.xml, server.xml, and serverindex.xml files from install_root/config.

    If instructed to do so by support, take a security and SAS trace of the problem in addition to the information requested above.

    Notes:
    *If SAS traces are requested by support, ORB tracing is usually also required. The instructions below include instructions for security, SAS and ORB tracing. If SAS tracing is not requested, you can eliminate the ORBRas=all=enabled and SASRas=all=enabled strings, as well as the CORBA arguments in the Generic JVM argument field.

    *Problems with JAAS login require security and SAS tracing only.

    1. Stop the application server and the nodeagent. Leave the deployment manager running.

    2. Enable tracing on the application server, nodeagent, and deployment manager in the administrative console.
      1. For the application server, go to Servers > Application Servers>server_name> Diagnostic Trace Service.

      2. For the node agent, go to System Administration > Node agents > Nodeagent server > Diagnostic Trace Service.

      3. For the deployment manager, go to System Administration > Deployment manager > Diagnostic Trace Service.

      For each process a. through c. above, specify the following trace string and an output file name in Diagnostic Trace Service:

      SASRas=all=enabled:com.ibm.ws.security.*=
      all=enabled:ORBRas=all=enabled

      Also for each process a. through c. above, select Process Definition Java Virtual Machine and insert the following in the Generic JVM arguments field:

      -Dcom.ibm.CORBA.Debug=true
      -Dcom.ibm.CORBA.CommTrace=true

    3. Save the changes and remember to synch with nodes.

    4. Stop the deployment manager.

    5. Delete or rename the existing logs in install_root/logs directories.

    6. Start the deployment manager, node agent, and application server.

    7. Recreate your problem. It is very important to note the time the problem occurs, the user ID, and the URL being invoked.

    8. Run the collector tool in install_root/bin to collect all logs and system configuration for the application server. If you are running network deployment, also run the collector tool in dmgr_install_root/bin. See the link to collector tool instructions in the required files section above. Collect both JAR files.

    9. Follow instructions tosend diagnostic information to IBM support.

    For a listing of all technotes, downloads, and educational materials specific to the Security component, search the WebSphere Application Server support site.

Related information
Steps to getting support

Submitting information to IBM support

Collector tool for WebSphere 5.0.x

Collector tool for WebSphere 5.1.x

Troubleshooting guide for security in WebSphere 5.0.x

Troubleshooting Single Sign On: WebSphere 5.1.x

Troubleshooting guide for security in WebSphere 5.1.x

Problem Determination whitepaper: WebSphere 4.0.x












Document Information

Product categories: Software, Application Servers, Distributed Application & Web Servers, WebSphere Application Server, Security
Operating system(s): AIX, HPUX, Linux, Multi-Platform, Solaris, Windows
Software version: 3.5, 4.0, 5.0, 5.1, 6.0
Software edition: Edition Independent
Reference #: 1140669
IBM Group: Software Group
Modified date: 2004-12-16