Setting up Network Channel Bonding (Load Balancing / Active-Standby) on Red Hat

Notice: The article is for Red Hat Enterprise Linux (RHEL) 5.7 and 6.2. It may also suitable for other newer version of Red Hat.

This article is inspired by this article.

Situation

  • You have a fresh installed server running RHEL
  • The server has two Ethernet interface eth0 and eth1
  • The server has no bond device set up before
  • You want to configure eth0 and eth1 to be load balancing or active-standby
  • You will use static IP for the network connection

Step 1: Add a bond device configuration file

2012-06-11 Edit 1: Till now I am still wondering if MACADDR or HWADDR should be specified in the configuration file because in sample configuration file in the Red Hat official document here it does not contain such fields. At this stage I would recommend not specifying both MACADDR and HWADDR unless you run into physical address conflict problem.

2012-06-11 Edit 2: After going through the official Red Hat document for RHEL5 here, the correct way to specify the bonding options should be inside the ifcfg-bond0 file.

Create a new file in:

/etc/sysconfig/network-scripts/ifcfg-bond0

with the following content:
For MAC address, please assign a value on your own so that it won’t conflict with other MAC address on the network. The main reason why MAC address needed to specified is that bond0 by default will use the hardware address of eth0. When eth1 is down, the hardware address of eth0 will conflict with the MAC address of bond0, which we don’t want to see. Please notice that it should be MACADDR but not HWADDR in the bond configuration file.

DEVICE=bond0
ONBOOT=yes
BOOTPROTO=none
MACADDR=A1:B8:E8:08:5E:FD
IPADDR=192.168.0.30
NETMASK=255.255.255.0
GATEWAY=192.168.0.1
USERCTL=no
BONDING_OPTS="mode=balance-alb miimon=100"

Remember to replace the value for IP address, netmask and gateway.

For active-standby channel bonding, you should use the following instead:

mode=active-backup miimon=100

The miimon options is to enable monitoring of the link status. You may refer to the Red Hat official documentation here or another reference here for more parameters and options available for configuration.

For RHEL6, you may also want to include the following line in the file to avoid getting the error Device not managed by NetworkManager when activating the interface.

NM_CONTROLLED=no

Step 2: Modify existing Ethernet device configuration files

2012-06-11 Edit 3: See 2012-06-11 Edit 1 for the reason of removing the line HWADDR.

Modify the configuration file. Remember to replace the HWADDR with the original hardware address from the Ethernet interface.

Modify the file

/etc/sysconfig/network-scripts/ifcfg-eth0

with the following content:

DEVICE=eth0
BOOTPROTO=none
HWADDR=6C:54:82:06:27:A6
ONBOOT=yes
USERCTL=no
MASTER=bond0
SLAVE=yes

Do the same for

/etc/sysconfig/network-scripts/ifcfg-eth1

with the following:

DEVICE=eth1
BOOTPROTO=none
HWADDR=35:52:AB:59:AB:64
ONBOOT=yes
USERCTL=no
MASTER=bond0
SLAVE=yes

For RHEL6, you may also want to include the following line in the file to avoid getting the error Device not managed by NetworkManager when activating the interface.

NM_CONTROLLED=no

Step 3: Configure bond module

2012-06-11 Edit 4: See 2012-06-11 Edit 2 for the reason why option bond0 is deleted.

For RHEL5, append to file:

/etc/modprobe.conf

For RHEL6, create a file:

/etc/modprobe.d/bonding.conf

with the following one line two lines:

For load balancing:

alias bond0 bonding
options bond0 mode=balance-alb miimon=100

For active-standby:

alias bond0 bonding
options bond0 mode=active-backup miimon=100

The miimon options is to enable monitoring of the link status. You may refer to here for more parameters and options available for configuration.

Step 4: Enable bonding module

To enable bonding module, simply restart your server or execute the two commands

modprobe -v bonding

service network restart

Step 5: Check the status of bonding channel

Execute the command

cat /proc/net/bonding/bond0

to check the status. Below is the sample output:

Bonding Mode: load balancing (round-robin)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 200
Down Delay (ms): 200

Slave Interface: eth0
MII Status: up
Link Failure Count: 0
Permanent HW addr: 6c:54:82:06:27:a6

Slave Interface: eth1
MII Status: up
Link Failure Count: 0
Permanent HW addr: 35:52:ab:59:ab:64

The channel bonding has been set up and running. However if you are setting up a new server you might be interested in setting up your DNS as well.

Extra Step 1: Set up DNS

Edit the file

/etc/resolv.conf

and append a line prefixed with nameserver for each DNS server. Remember to replace the IP of the DNS server.

nameserver 8.8.8.8

Extra step 2: Checking if DNS working

Simply execute command like

ping www.google.com

and see if the it is successfully resolved into IP address for ping. You should see something like this.

[root@server ~]# ping www.google.com
PING www.l.google.com (72.14.203.99) 56(84) bytes of data.

Note: Problem with MAC address flip flopping with load balancing

If you are using the load balancing channel bonding, your router may complaint that the MAC address is flip flopping because the same MAC address is appearing in two different links. If the use of load balancing is purely for redundancy, consider using active standby mode. If not, most likely you have to work on your router configuration.

About these ads

10 responses to “Setting up Network Channel Bonding (Load Balancing / Active-Standby) on Red Hat

  1. Very good information. Upto the point.

    After this configuration, I have tested the failover and it works. How to test that load balancing is working between the two network ports.

    Thank you,

    Anil.

    • Thanks for your comment!

      Actually I don’t have an idea on this neither. Too bad I don’t have the environment to try that out (since the servers has been configured as active-standby now).

      Anyway I guess you can try checking using the ifconfig command to see the total amount of data flowed over the network interface. Or if the connecting switch is powerful enough you maybe able to see the bandwidth utilization of the ports. Do remember to tell me if it is working or not, thanks!

  2. The bonding is done for eth1 and eth2. If we see the RXbytes and TX bytes of bond1, it is addition of eth1 and eth2 RXbytes and TX bytes. Does this prove the load balancing is working?

    $ /sbin/ifconfig
    bond1     Link encap:Ethernet  HWaddr D4:BE:D9:B6:DB:6D
              inet addr:192.168.252.37  Bcast:192.168.252.255  Mask:255.255.255.0
              inet6 addr: fe80::d6be:d9ff:feb6:db6d/64 Scope:Link
              UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
              RX packets:5614205 errors:0 dropped:0 overruns:0 frame:0
              TX packets:6967641 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:0
              RX bytes:628109973 (599.0 MiB)  TX bytes:7794190802 (7.2 GiB)
    
    eth1      Link encap:Ethernet  HWaddr D4:BE:D9:B6:DB:6D
              UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
              RX packets:2738093 errors:0 dropped:0 overruns:0 frame:0
              TX packets:3482902 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:1000
              RX bytes:313429711 (298.9 MiB)  TX bytes:3895891932 (3.6 GiB)
              Interrupt:48 Memory:d6000000-d6012800
    
    eth2      Link encap:Ethernet  HWaddr D4:BE:D9:B6:DB:6D
              UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
              RX packets:2876112 errors:0 dropped:0 overruns:0 frame:0
              TX packets:3484739 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:1000
              RX bytes:314680262 (300.1 MiB)  TX bytes:3898298870 (3.6 GiB)
              Interrupt:32 Memory:d8000000-d8012800
    • Yes that is! TX bytes refer to the transferred byte from the interface while RX bytes refer to the received byte. If you are still not sure, you can try transferring a large file, say 100MB, and check if the TX bytes or RX bytes of eth1 and eth2 actually increase at the same time and sum up to have 100MB! (and sure you may want to reset the statistic (you may try restarting the network interface to do that) or do a file receive in the server since 100MB adding to the existing 3.6GB TX bytes won’t be so distinctive)

  3. I could able to do all the configuration for the eth0,eth1,bond0
    but NO INTERNET.
    do i have to use the bond0 ip on another system or the same linux systems gets the internet..
    BTW,i am new to linux please bear ….!!

    • Sorry I am not quite sure what you said.

      I would suggest you first fallback to use 1 interface (eth0 only) to see if you can get on the Internet first. If it does, then you can setup the channel bonding using the same IP configuration information. I guess you are using static IP right?

      And are you using active-standby channel bonding? You might went into some problem if your network configuration is not properly set up if you use load balancing.

    • I haven’t try that but I think it is possible. You will need to create two network interfaces on the VM and then configure active-standby in the guest OS (Redhat).
      I would be good to try out active-standby connection in that way.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s