MustGather: Edge Load Balancer problems in Edge Server (Network Dispatcher)
 Technote (troubleshooting)
 
Problem(Abstract)
MustGather for problems with the WebSphere® Application Server Edge component, Load Balancer. Gathering this information before calling IBM support will help familiarize you with the troubleshooting process and save you time.
 
Resolving the problem
If you have already contacted support, continue on to the component-specific MustGather information. Otherwise, click: MustGather: Read first for all WebSphere Application Server Products.
Load Balancer specific MustGather information
  1. Configuration files from the Load Balancer machines. Enter the following command to save the current configuration:

    dscontrol file save primary.cfg

    Note: The primary.cfg file is saved in the configurations directory.

  2. Exact version of Load Balancer installed:
    1. Issue the command:

      dscontrol executor report

    2. If Load Balancer is not running, issue the following:
      • For AIX®:

        lslpp -l | grep intnd

        Note: Use "grep ibmlb" for version 5.

      • For Solaris:

        pkginfo -l ibmndadm

        Note: Use "ibmlbadm" for version 5.

      • For Linux®:

        rpm -qa | grep ibmnd

      • For Microsoft® Windows®:

        dscontrol executor report

  3. The log files from the Load Balancer. The logs are more helpful if the LogLevel is set to 5 when data is collected. Provide the files from the logs directory, which by default is located in the following directory:

    ...ibm/edge/lb/servers/logs/dispatcher/

    Note: The main log file needed is the server.log file, but it is best to zip the entire logs directory and send all of it to IBM Support.

  4. Operating system and Java™ version. From a command prompt, enter the following:

    java -fullversion

  5. List network interfaces from the Load Balancer machine and backend servers:
    • For UNIX®:

      netstat -ni

    • For Windows:

      ipconfig

  6. List active routes from Load Balancer machine and backend servers:
    • For UNIX:

      netstat -nr

    • For Windows:

      route print

  7. List active connections from Load Balancer machine and backend servers:

    netstat -an

  8. List interface settings from Load Balancer machine and backend servers:
    • For UNIX only:

      ifconfig -a

  9. Follow instructions to send diagnostic information to IBM support.


In addition to the preceding documentation, collect the following, if applicable:
  • High availability problems:
    1. The hamon.log file at loglevel 5:

      dscontrol set loglevel 5

    2. The reach.log file at loglevel 5:

      dscontrol manager reach set loglevel 5

    3. All script files, which are located in the following directory:
      • For UNIX:

        /opt/ibm/edge/lb/servers/bin

      • For Windows:

        c:\Program Files\ibm\edge\lb\servers\bin

  • Advisor problems:
    1. Advisor log at loglevel 5:

      dscontrol advisor loglevel <protocol> <port> 5

      The output will be of the format, <advisor name>_<port>.log

    2. Log files located in the following Edge Load Balancer directory:

      dispatcher/logs

  • "Not able to hit the cluster" error message:
    1. On the same subnet, NOT on a Load Balancer machine:
      1. "Ping" the cluster address.

      2. Enter the following command:

        arp -a

      3. Look through the arp output and match the MAC address to the output of a netstat -ni command to see which machine physically owns the cluster address.

    2. Make sure your cluster is aliased on the loopback of the backend servers if using MAC based forwarding. For Windows and AIX® V5 (all releases), check for an extra route in the routing table and delete it if one exists.

  • "All else fails...":
    Collect a trace of your network traffic:
    • For AIX, from your Load Balancer machine:
      1. Enter the command:

        iptrace -a -s failing_client_ip -d cluster_ip -b iptrace.trc

      2. Recreate your problem.

      3. Kill the process.

      4. Do not format the trace. Send the raw, unformatted iptrace file.
    • For Solaris:

      snoop -v client_ip destination_ip snooptrace.out

    • For Windows: Use sniffer, Network Monitor, or Ethereal trace.

    • For Linux:

      tcpdump -w trace.out


For a listing of all technotes, downloads, and educational materials specific to the Edge component, Load Balancer, search the WebSphere Application Server support site or the WebSphere Edge Server support site.
 
Related information
Submitting information to IBM support
Steps to getting support
MustGather: Read first
Troubleshooting guide
 
 
Cross Reference information
Segment Product Component Platform Version Edition
Application Servers WebSphere Application Server Edge Component 6.1, 6.0
 
 


Document Information


Product categories: Software > Application Servers > Distributed Application & Web Servers > WebSphere Application Server > Edge Component
Operating system(s): Windows
Software version: 6.1
Software edition:
Reference #: 1140797
IBM Group: Software Group
Modified date: Jan 29, 2007