Configuring high availability

The high availability feature involves the use of a second Dispatcher machine. The first Dispatcher machine performs load balancing for all the client traffic as it does in a single Dispatcher configuration. The second Dispatcher machine monitors the "health" of the first, and takes over the task of load balancing if it detects that the first Dispatcher machine has failed.

About this task

When you configure high availability, each of the two machines is assigned a specific role, either primary or backup. The primary machine sends connection data to the backup machine on an ongoing basis. While the primary is active (load balancing), the backup is in a standby state, continually updated and ready to take over, if necessary.

The communication sessions between the two machines are referred to as heartbeats. The heartbeats allow each machine to monitor the health of the other. If the backup machine detects that the active machine has failed, it will take over and begin load balancing. At that point the statuses of the two machines are reversed: the backup machine becomes active and the primary becomes standby.
Note: In the high availability configuration, both primary and backup machines must be on the same subnet with identical configuration.

For the complete syntax see dscontrol highavailability. For a more complete discussion of many of the tasks below, see Configuring the Dispatcher machine.

Tips for configuring high availability:

  1. To configure a single Dispatcher machine to route packets without a backup, do not issue any of the high availability commands at startup.
  2. To convert two Dispatcher machines configured for high availability to one machine running alone, stop the executor on one of the machines, then delete the high availability features (the heartbeats, reach, and backup) on the other.
  3. [Linux] Linux for s/390: In both of the two cases above, you must alias the network interface card with cluster addresses, as required.
  4. When running two Dispatcher machines in a high availability configuration, unexpected results can occur if you set any of the parameters for the executor, cluster, port, or server (for example, port stickytime) to different values on the two machines.

Procedure

  1. [Linux] If you are running Linux for s/390 operating systems in layer 3 mode, create alias script files on each of the two Dispatcher machines. See Scripts to run with high availability for more information on the available scripts. The scripts you create should contain commands to:
    • Configure the cluster ip address on the interface
    • Add an iptables rule to drop incoming packets destined to the cluster address
    See the topic Configuring the Dispatcher machine for more information.
  2. Start the server on both Dispatcher server machines.
  3. Start the executor on both machines.
  4. Ensure that the non-forwarding address (NFA) of each Dispatcher machine is configured, and is a valid IP address for the subnet of the Dispatcher machines.
  5. Add the heartbeat information on both machines:
    dscontrol highavailability heartbeat add source_address destination_address
    Source_address and destination_address are the IP addresses (either DNS names or IP addresses) of the Dispatcher machines. The values will be reversed on each machine. For example:
    Primary - highavailability heartbeat add 9.67.111.3 9.67.186.8
    Backup - highavailability heartbeat add 9.67.186.8 9.67.111.3 
    At least one heartbeat pair must have the NFAs of the pair as the source and destination address. If possible, at least one of the heartbeat pairs should be across a separate subnet than the regular cluster traffic. Keeping the heartbeat traffic distinct will help prevent false takeovers during very heavy network loads and also improve complete recovery times after a failover.
    1. Optional: Set the number of seconds that the executor uses to timeout high availability heartbeats. The default is 2 seconds. For example:
      dscontrol executor set hatimeout 3
  6. On both machines, configure the list of IP addresses that the Dispatcher must be able to reach in order to ensure full service, using the reach add command. Reach targets are recommended but not required. See Detecting server failures with heartbeats and reach targets for more information. For example:
     dscontrol highavailability reach add 9.67.125.18 
  7. Add the backup information to each machine:
    1. For the primary machine:
      dscontrol highavailability backup add primary [auto | manual] port
    2. For the backup machine:
       dscontrol highavailability backup add backup [auto | manual] port
      Note: Select an unused port on your machines as the port. The port number entered will be used as a key to ensure the correct host is receiving the packet.
  8. Check the high availability status on each machine:
     dscontrol highavailability status

    The machines should each have the correct role (backup or primary) and states. The primary should be active; the backup should be in standby mode. The recovery strategies must be the same.

  9. Set up the cluster, port, and server information on both machines.
  10. Optional: Enable replication of connection and affinity records.
    New feature New feature:
    This feature provides the ability to replicate connection and affinity records between high availability partners. When the records are replicated, connection and affinity states are preserved so that the connections can continue even after takeover has taken place.newfeat
    • Enable replication for connection and affinity records:
      dscontrol port set cluster@port repstrategy both
    • Enable replication for only connection records:
      dscontrol port set cluster@port repstrategy connection
    • Enable replication for only affinity records:
      dscontrol port set cluster@port repstrategy affinity
    • To disable replication:
      dscontrol port set cluster@port repstrategy none
    For more information on this command, refer to the topic on the dscontrol port command.
  11. [Linux] Optional: Suppress the unreachable packets for the ICMP port that are generated by the operating system in response to heartbeat packets
    # iptables -t filter -A INPUT -p udp --destination-port <port> -j DROP
    <port> is the port number for the heartbeat. For more information on this issue, read the technote ICMP Port unreachable sent when HA packet received.
  12. Start the manager and advisors on both machines.
Task topic    

Terms and conditions for information centers | Feedback

Last updated: April 16, 2014 11:59 AM EDT
File name: tprf_highavail.html