Tarrasque

This Howto explain how to install the Bonding or the Teaming solution on a RedHat distribution. One of the goal of the teaming and bonding solution is to use all networks interfaces for backup or round robin solution.
The bonding need to work with a minimum of 2 NIC to have real goal.
The concept of NIC Bonding (or sometimes called NIC Teaming) is that you have two NICs bonded together to create only one physical device.

Bonding schema and switch topology :

| |
|port3 port3|
+-----+----+ +-----+-----+
| |port2 port2 | |
| switch A +---------------------------------+ switch B |
| | | |
+-----+----+ +-----+-----+
|port1 port1|
| +-------+ |
+-----------------+ host1 +------------------+
eth0 +-------+ eth1

Before attempting to bond two NICs, i recommended to verify the integrity and functionality of each NIC. Please check your interface status with ethtool:
#ethtool eth0
Settings for eth0:
Supported ports: [ TP MII ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
Advertised auto-negotiation: Yes
Speed: 100Mb/s
Duplex: Full
Port: MII
PHYAD: 1
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: g
Wake-on: g
Current message level: 0x00000007 (7)
Link detected: yes

#ethtool eth1
Settings for eth1:
Supported ports: [ TP MII ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
Advertised auto-negotiation: Yes
Speed: 100Mb/s
Duplex: Full
Port: MII
PHYAD: 1
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: g
Wake-on: g
Current message level: 0x00000007 (7)
Link detected: yes
Now the Interface seems to be good for the configuration, we have to ckeck if you have all kernel modules.
#modprobe --list | grep bonding
/lib/modules/2.6.*/kernel/drivers/net/bonding/bonding.ko
#modprobe --list | grep mii
/lib/modules/2.6.*/kernel/drivers/net/mii.ko
Your server seems to be ready fot the configuration of the module for a backup connection:
#vi /etc/modprobe.conf
alias bond0 bonding
options bond0 miimon=80 mode=1
To have more information about the bonding module you have to do this commande line.
In bold font you have the main status for the bonding function.
#modinfo bonding
author: Thomas Davis, tadavis@lbl.gov This e-mail address is being protected from spambots, you need JavaScript enabled to view it and many others
description: Ethernet Channel Bonding Driver, v3.1.2
version: 3.1.2
license: GPL
srcversion: 6CD19765D6431C07199456E
depends:
vermagic: 2.6.18-53.1.4.el5xen SMP mod_unload gcc-4.1
parm: max_bonds:Max number of bonded devices (int)
parm: miimon:Link check interval in milliseconds (int)
parm: updelay:Delay before considering link up, in milliseconds (int)
parm: downdelay:Delay before considering link down, in milliseconds (int)
parm: use_carrier:Use netif_carrier_ok (vs MII ioctls) in miimon; 0 for off, 1 for on (default) (int)
parm: mode:Mode of operation : 0 for balance-rr, 1 for active-backup, 2 for balance-xor, 3 for broadcast, 4 for 802.3ad, 5 for balance-tlb, 6 for balance-alb (charp)
parm: primary:Primary network device to use (charp)
parm: lacp_rate:LACPDU tx rate to request from 802.3ad partner (slow/fast) (charp)
parm: xmit_hash_policy:XOR hashing method: 0 for layer 2 (default), 1 for layer 3+4 (charp)
parm: arp_interval:arp interval in milliseconds (int)
parm: arp_ip_target:arp targets in n.n.n.n form (array of charp)
parm: arp_validate:validate src/dst of ARP probes: none (default), active, backup or all (charp)
module_sig: 883f350474fb1b5bf23bf4d191725311298f009e2cbe4ce27b13fd747fc988615153453d50a29aff0a08158d98a7570129eb3197be02aebeaf52d9aebe
miimon is use for the monitoring of each ethX each 80 milliseconds, but you can use also a arp_ip_target like this other configuration for this module.
#vi /etc/modprobe.conf
alias bond0 bonding
options bond0 mode=1 arp_ip_target=192.168.1.1 arp_interval=200 primary=eth0
This configuration use arp resquest on the primary interface each 200ms
Now the kernel module is configure and we have to load thoses modules :
#modprobe bonding
#modprobe mii
All modules are loaded and ready for the configuration of each NIC
Configuration of eth0:
#vi /etc/sysconfig/network-script/ifcfg-eth0
DEVICE=eth0
BOOTPROTO=none
ONBOOT=yes
MASTER=bond0
SLAVE=yes
USERCTL=no
Configuration of eth1
#vi /etc/sysconfig/network-script/ifcfg-eth1
DEVICE=eth1
BOOTPROTO=none
ONBOOT=yes
MASTER=bond0
SLAVE=yes
USERCTL=no
Configuration of bond0
#vi /etc/sysconfig/network-script/ifcfg-bond0
DEVICE=bond0
BOOTPROTO=none
ONBOOT=yes
NETMASK=255.255.255.0
IPADDR=192.168.1.5
USERCTL=no
Now to finish the installation we have to applied the network parameters.
#/etc/init.d/network restart
Your bonding is now operational.
We have to test the bonding interface :
# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.1.2 (January 20, 2007)
Bonding Mode: fault-tolerance (active-backup)
Primary Slave: None
Currently Active Slave: eth0
MII Status: up
MII Polling Interval (ms): 80
Up Delay (ms): 0
Down Delay (ms): 0
Slave Interface: eth0
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:XX:XX:XX:XX:c2
Slave Interface: eth1
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:XX:XX:XX:XX:c4

Tarrasque

Linux Teaming/Bonding (合併網卡) 實作
目的

• 網卡自動備援
• 增加傳輸頻寬
• 多網卡負載平衡

實作環境

• Fedora Core 3 Linux
• Realtek RTL8139 PCI Fast Ethernet NIC * 2 (driver: 8139too)
• LAN: 192.168.0.0/24; Linux Host: 192.168.0.1; Gateway: 192.168.0.254

# vi /etc/sysconfig/network-scripts/ifcfg-bond0
DEVICE=bond0
IPADDR=192.168.0.1
NETMASK=255.255.255.0
ONBOOT=yes
BOOTPROTO=none
USERCTL=no

# vi /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none
HWADDR=xx:xx:xx:xx:xx:xx

# vi /etc/sysconfig/network-scripts/ifcfg-eth1
DEVICE=eth1
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none
HWADDR=xx:xx:xx:xx:xx:xx
vi /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=myhost
GATEWAY=192.168.0.254

# vi /etc/modprobe.conf
alias bond0 bonding
options bond0 miimon=100 mode=1

手動啟用 bonding (或於完成上述步驟後重新啟動電腦)

# service network stop
modprobe bonding miimon=100 mode=1
ifconfig bond0 192.168.0.1 netmask 255.255.255.0
ifenslave bond0 eth0 eth1
route add default gw 192.168.0.254 dev bond0

查看 bonding 狀態
# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v2.6.0 (January 14, 2004)
Bonding Mode: fault-tolerance (active-backup)
Primary Slave: none
Currently Active Slave: eth0
MII Status: up
MII Polling Internal (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Slave Interface: eth0
MII Status: up
Link Failure Count: 0
Permanent HW addr: xx:xx:xx:xx:xx:xx
Slave Interface: eth1
MII Status: up
Link Failure Count: 0
Permanent HW addr: xx:xx:xx:xx:xx:xx
bonding modes
OR
# modinfo bonding
author: Thomas Davis, tadavis@lbl.gov This e-mail address is being protected from spambots, you need JavaScript enabled to view it and many others
description: Ethernet Channel Bonding Driver, v3.1.2
version: 3.1.2
license: GPL
srcversion: 6CD19765D6431C07199456E
depends:
vermagic: 2.6.18-53.1.4.el5xen SMP mod_unload gcc-4.1
parm: max_bonds:Max number of bonded devices (int)
parm: miimon:Link check interval in milliseconds (int)
parm: updelayelay before considering link up, in milliseconds (int)
parm: downdelayelay before considering link down, in milliseconds (int)
parm: use_carrier:Use netif_carrier_ok (vs MII ioctls) in miimon; 0 for off, 1 for on (default) (int)
parm: mode:Mode of operation : 0 for balance-rr, 1 for active-backup, 2 for balance-xor, 3 for broadcast, 4 for 802.3ad, 5 for balance-tlb, 6 for balance-alb (charp)
parm: primary:Primary network device to use (charp)
parm: lacp_rate:LACPDU tx rate to request from 802.3ad partner (slow/fast) (charp)
parm: xmit_hash_policy:XOR hashing method: 0 for layer 2 (default), 1 for layer 3+4 (charp)
parm: arp_interval:arp interval in milliseconds (int)
parm: arp_ip_target:arp targets in n.n.n.n form (array of charp)
parm: arp_validate:validate src/dst of ARP probes: none (default), active, backup or all (charp)
module_sig: 883f350474fb1b5bf23bf4d191725311298f009e2cbe4ce27b13fd747fc988615153453d50a29aff0a08158d98a7570129eb3197be02aebeaf52d9aebe
miimon is use for the monitoring of each ethX each 80 milliseconds, but you can use also a arp_ip_target like this other configuration for this module.


0 = balance-rr
‧負載平衡模式, 需有 switch 設定 (trunk) 支援才能發揮實質效果
‧具容錯功能, 其中一張 Slave 網卡失效仍可持續運作
Round-robin policy: Transmit packets in sequential order from the first available slave through the last. This mode provides load balancing and fault tolerance.


1 = active-backup
‧同一時間只有單一 Slave 網卡運作
‧Active Slave 網卡失效時自動啟用次一順位 Slave 網卡
‧不需 switch 支援
Active-backup policy: Only one slave in the bond is active. A different slave becomes active if, and only if, the active slave fails. The bond's MAC address is externally visible on only one port (network adapter) to avoid confusing the switch. This mode provides fault tolerance. The primary option affects the behavior of this mode.


2 = balance-xor
*未研究*
XOR policy: Transmit based on [(source MAC address XOR'd with destination MAC address) modulo slave count]. This selects the same slave for each destination MAC address. This mode provides load balancing and fault tolerance.


3 = broadcast
‧所有 Slave 網卡一齊收送網路封包
‧具容錯功能, 其中一張 Slave 網卡失效仍可持續運作
Broadcast policy: transmits everything on all slave interfaces. This mode provides fault tolerance.


4 = 802.3ad
*未研究*
IEEE 802.3ad Dynamic link aggregation. Creates aggregation groups that share the same speed and duplex settings. Utilizes all slaves in the active aggregator according to the 802.3ad specification.
Pre-requisites:
1. Ethtool support in the base drivers for retrieving the speed and duplex of each slave.
2. A switch that supports IEEE 802.3ad Dynamic link aggregation.
Most switches will require some type of configuration to enable 802.3ad mode.


5 = balance-tlb
‧傳出自動負載平衡
‧傳入由 Current Active Slave 負責
‧具容錯功能, 其中一張 Slave 網卡失效仍可持續運作
‧不需 switch 支援及設定
Adaptive transmit load balancing: channel bonding that does not require any special switch support. The outgoing traffic is distributed according to the current load (computed relative to the speed) on each slave. Incoming traffic is received by the current slave. If the receiving slave fails, another slave takes over the MAC address of the failed receiving slave.


6 = balance-alb
‧傳出及傳入皆自動負載平衡
‧具容錯功能, 其中一張 Slave 網卡失效仍可持續運作
‧Slave 網卡 driver 需支援 setting hardware address 功能
‧不需 switch 支援及設定
Adaptive load balancing: includes balance-tlb plus receive load balancing (rlb) for IPV4 traffic, and does not require any special switch support. The receive load balancing is achieved by ARP negotiation. The bonding driver intercepts the ARP Replies sent by the local system on their way out and overwrites the source hardware address with the unique hardware address of one of the slaves in the bond such that different peers use different hardware addresses for the server.
Prerequisite:
Ethtool support in the base drivers for retrieving the speed of each slave.


備註 - 關於螃蟹卡 (8139too):
無法支援 mode 6 (balance-alb), 錯誤訊息如下
bonding: Error: dev->set_mac_address of dev eth0 failed! ALB mode requires that the base driver support setting the hw address also when the network device's interface is open