Auto-negotiation

By design, Ethernet network interface cards (NIC) automatically negotiate the best speed and duplex with the switch that it is connected to. Generally, auto-negotiation work. We have, however, seen many cases where auto-negotiation drops the connection to sub-optimal levels (e.g., 10mps half-duplex) after a server boot.

The impact of an incorrectly negotiated network card is dramatic. For example, at one customer, application servers took over 20 minutes to start when the network card on the administration server negotiated the wrong settings.

If both the network interface card and the switch are capable of full-duplex 100mbps or 1000mbps, you can let then auto-negotiate. Alternatively, you can manually set the higher speed and duplex as described in subsequent "Auto-negotiation" sections.

An easy way to check the NIC negotiation is to FTP, SCP or RCP a large file (e.g., 256MB) file from a test node to all other nodes.

From the database server node, create a 256MB file using the following command:

   dd if=/dev/zero of=/tmp/egg bs=16384 count=16384
   

Assuming that you have three nodes (applservernode1 to applservernode3) and you can rcp or scp into each node, issue the following:

   export ALLHOSTS="applservernode1 applservernode2 applservernode3"
   for i in $ALLHOSTS
   do
   time rcp /tmp/egg $i:/tmp/egg
   done
    

If you cannot rcp or scp, you can issue an FTP transfer.

The time to transfer the 256MB file should be around 20 seconds for 100mbps Fast Ethernet and around 5 seconds for 1Gbps networks (for FTP). You likely have a network negotiation problem if the transfer times are much slower (for example 200 seconds).

Please see the following sections on how to monitor and set the network speed and bandwidth: