Adding LAG and LACP

Owned by Tracy Yang
Nov 13, 2023
5 min read

LAG/LACP 

Link aggregation (LAG) refers to aggregating multiple physical network links in parallel in order to increase throughput beyond what a single link could sustain, and provide redundancy in case any of the links should fail.

Link Aggregation Control Protocol (LACP) refers to bundling of several physical ports together to form a single logical channel. Network devices use LACP to negotiate automatic bundling of links by sending LACP packets to its directly connected peer(s)

LACP features and practical examples:

  1. Maximum number of bundled ports allowed in the port channel:  valid values are from 1 to 8.
  2. LACP packets are sent with a well known multicast group MAC address 01:80:c2:00:00:02
  3. LACP detection period:
    • LACP packets are transmitted every second
    • Keep alive mechanism for link members: (default: slow = 30s, fast=1s)
  4. LACP can have the port-channel load-balance mode:
    • load-balancing can be configured using a combination of in_port, L2, L3 and L4 headers.
  5. LACP mode:
    • active: Enables LACP unconditionally.
    • passive: Enables LACP only when an LACP device is detected. (This is the default state)

 

1, PicOS OVS supports LAG and LACP.

2, PicOS can support 48 LAG or LACP at most. Each LAG has a maximum of 8 member ports.

3, The port ranges in PicOS are as follows:

Port TypePort Number

LAG

1025-2047


Create a Static LAG

In following configuration, user can create LAG ae1, and add port 2 and port 3 into this LAG

root@PicOS-OVS# ovs-vsctl  add-port br0 ae1 vlan_mode=trunk tag=1 -- set Interface ae1 type=pica8_lag
root@PicOS-OVS# ovs-vsctl -- set Interface ae1 options:lag_type=static
root@PicOS-OVS# ovs-vsctl -- set Interface ae1 options:members=ge-1/1/2,ge-1/1/3

 

Dual Homing a Host

While dual-homing a host does not require a LAG configuration on the switch, this configuration illustrates steps required to setup LAG on the host connected to two different switches.

 

Host LAG ConfigurationSwitch Port Configuration

pica8@Ubuntu$ more /etc/modprobe.d/bonding.conf 

bonding mode=1 miimon=100 

pica8@Ubuntu$ more /etc/network/interfaces

 

# NIC bonding config for LAG active-backup START

auto eth1

iface eth1 inet manual

        bond-master bond0

 bond-primary eth1

auto eth2

iface eth2 inet manual

        bond-master bond0

auto bond0

iface bond0 inet static

 address 10.0.0.20

 netmask 255.255.255.0

bond-mode active-backup

bond-miimon 100

bond-slaves none 

# NIC bonding config for LAG active-backup END

 

pica8@Ubuntu$  cat /proc/net/bonding/bond0 

Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

 

Bonding Mode: fault-tolerance (active-backup)

Primary Slave: eth1 (primary_reselect always)

Currently Active Slave: eth1

MII Status: up

MII Polling Interval (ms): 100

Up Delay (ms): 0

Down Delay (ms): 0

 

Slave Interface: eth2

MII Status: up

Speed: 1000 Mbps

Duplex: full

Link Failure Count: 0

Permanent HW addr: 00:0c:29:70:2e:c2

Slave queue ID: 0

 

Slave Interface: eth1

MII Status: up

Speed: 1000 Mbps

Duplex: full

Link Failure Count: 0

Permanent HW addr: 00:0c:29:70:2e:b8

Slave queue ID: 0

OVS-Sw1$ovs-vsctl add-port br0 ge-1/1/1 vlan_mode=trunk tag=1 -- set interface ge-1/1/1 type=pica8

 

OVS-Sw2$ovs-vsctl add-port br0 ge-1/1/1 vlan_mode=trunk tag=1 -- set interface ge-1/1/1 type=pica8


Create a LACP Port

Following is the configuration on the host (running Ubuntu):

pica8@Ubuntu:~$ more /etc/modprobe.d/bonding.conf 
bonding mode=4 miimon=100 lacp_rate=1

pica8@Ubuntu:~$ more /etc/network/interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface
auto eth0
iface eth0 inet dhcp

# NIC bonding config with LACP START
auto eth1
iface eth1 inet manual
        bond-master bond0
auto eth2
iface eth2 inet manual
        bond-master bond0
auto bond0
iface bond0 inet static
 address 10.0.0.10
 netmask 255.255.255.0
bond-mode 802.3ad 
bond-miimon 100
bond-lacp-rate 1
bond-slaves none 
# NIC bonding config with LACP END
 
 
pica8@Ubuntu:~$ sudo systemctl restart networking

pica8@Ubuntu:~$ cat /proc/net/bonding/bond0 
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)
Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
 
802.3ad info
LACP rate: fast
Min links: 0
Aggregator selection policy (ad_select): stable
Active Aggregator Info:
Aggregator ID: 3
Number of ports: 2
Actor Key: 17
Partner Key: 1
Partner Mac Address: 00:11:11:11:11:11
 
Slave Interface: eth1
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:0c:29:fd:75:ae
Aggregator ID: 3
Slave queue ID: 0
 
Slave Interface: eth2
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:0c:29:fd:75:b8
Aggregator ID: 3
Slave queue ID: 0

 

Following is the LACP configuration on the switch.

The default (slow) LACP timer is 30s whereas fast timer is 1s. LACP mode determines which node initiates the negotiation:  active sets the switch to initiate the LACP handshake, while in passive mode the switch accepts all incoming LACP requests.

admin@PicOS-OVS$ ovs-vsctl  add-port br0 ae1 vlan_mode=trunk tag=1 -- set Interface ae1 type=pica8_lag
admin@PicOS-OVS$ ovs-vsctl  -- set Interface ae1 options:lag_type=lacp
admin@PicOS-OVS$ ovs-vsctl  -- set Interface ae1 options:members=ge-1/1/1,ge-1/1/2
admin@PicOS-OVS$ ovs-vsctl  -- set Interface ae1 options:lacp-system-id=00:11:11:11:11:11
admin@PicOS-OVS$ ovs-vsctl  -- set Interface ae1 options:lacp-time=fast | slow


Optional Settings:
admin@PicOS-OVS$ ovs-vsctl  -- set Interface ae1 options:lacp-system-priority=32768
admin@PicOS-OVS$ ovs-vsctl  -- set Interface ae1 options:lacp-mode=active | passive
admin@PicOS-OVS$ ovs-vsctl  -- set Interface ge-1/1/2 options:lacp-port-id=2
admin@PicOS-OVS$ ovs-vsctl  -- set Interface ge-1/1/2 options:lacp-port-priority=32768
admin@PicOS-OVS$ ovs-vsctl  -- set Interface ge-1/1/2 options:lacp-aggregation-key=0
 


From PicOS version 2.7.1, support is added to configure minimum number port in a LACP or LAG. The command is as follows:

ovs-vsctl set Interface <lag port> options:lag_selected_min=<value>
admin@PicOS-OVS$ovs-vsctl set Interface ae1 options:lag_selected_min=1


Create Static Flow for LAG or LACP

In following configuration, user can create static flow whose output port is a LAG or LACP.

root@PicOS-OVS# ovs-ofctl add-flow br0 in_port=1025,actions=output:1 
root@PicOS-OVS# ovs-ofctl add-flow br0 in_port=1,actions=output:1025


LAG number index is shown as following:

LAG NameLAG Number
ae11025
ae21026
....
ae10232047

 

Display the LACP Information

User can display the LACP information with the following CLI commands.

admin@PicOS-OVS$ovs-appctl lacp/show

---- ae1 ----
status: active negotiated
sys_id:00:11:11:11:11:11
sys_priority: 32768
aggregation key: 1
lacp_time: fast
slave: ge-1/1/1: current attached
port_id: 1
port_priority: 32768
may_enable: true
actor sys_id:00:11:11:11:11:11
actor sys_priority:32768
actor port_id: 1
actor port_priority:32768
actor key: 1
actor state: activity timeout synchronized collecting distributing
partner sys_id:00:0c:29:fd:75:b8
partner sys_priority:65535
partner port_id: 2
partner port_priority:255
partner key: 17
partner state:activity timeout aggregation synchronized collecting distributing

 
slave: ge-1/1/2: current attached
port_id: 1
port_priority: 32768
may_enable: true
actor sys_id:00:11:11:11:11:11
actor sys_priority:32768
actor port_id: 1
actor port_priority:32768
actor key: 1
actor state: activity timeout synchronized collecting distributing
partner sys_id:00:0c:fe:ae:69:ac
partner sys_priority:65535
partner port_id: 2
partner port_priority:255
partner key: 17
partner state:activity timeout aggregation synchronized collecting distributing


LAG speed 

From PicOS version 2.7.2, support for LAG speed has been added. User can check the LAG speed configuration with the command "ovs-ofctl show br0 ".

Examples

Topology

ge-1/1/1------------connect ixia1

ge-1/1/2------------connect ixia2

ge-1/1/3------------connect ixia3

ge-1/1/4------------connect ixia4


Step 1: Create a bridge and add four ports (PX1,PX2,PX3)

ovs-vsctl add-br br0 -- set bridge br0 datapath_type=pica8
ovs-vsctl add-port br0 ge-1/1/1 vlan_mode=trunk tag=1 -- set interface ge-1/1/1 type=pica8
ovs-vsctl add-port br0 ge-1/1/2 vlan_mode=trunk tag=1 -- set interface ge-1/1/2 type=pica8
ovs-vsctl add-port br0 ge-1/1/3 vlan_mode=trunk tag=1 -- set interface ge-1/1/3 type=pica8
ovs-vsctl add-port br0 ge-1/1/4 vlan_mode=trunk tag=1 -- set interface ge-1/1/4 type=pica8

Step 2: Add a lag port

ovs-vsctl add-port br0 ae1 vlan_mode=trunk tag=1 -- set interface ae1 type=pica8_lag options:members=ge-1/1/2,ge-1/1/3,ge-1/1/4
ovs-vsctl -- set interface ae1 options:lag_type=static

Check the lag port speed

ovs-ofctl show br0
check "1025(ae1).*state.* LINK_UP"
check "speed.*300 Mbps"

Step 3: Add flow

ovs-ofctl add-flow br0 in_port=1,actions=output:1025
 check the flows:
ovs-ofctl dump-flows br0
check "in_port=1"
ovs-appctl pica/dump-flows
check "in_port=1"

Step 4: Send ipv4 packets and check the result

 (1) Send no changing packets to PX1,(dl_src=22:11:11:11:11:11,dl_dst=22:22:22:22:22:22,dl_vlan=10,ip,nw_src=192.168.1.10,nw_dst=192.168.2.10)

Result:

then PX2,PX3 or PX4 will transmit all the packets.
Check:
dl_src: 22 11 11 11 11 11
dl_dst:22 22 22 22 22 22
offset:30
ip:C0 A8 01 0A C0 A8 02 0A

(2)sending increasing src mac to PX1, (dl_src=22:11:11:11:11:11(increase),dl_dst=22:22:22:22:22:22,dl_vlan=10,ip,nw_src=192.168.1.10,nw_dst=192.168.2.10)

Result:

 

then both PX2, PX3 and PX4 will transmit the packets.
Check:
dl_dst:22 22 22 22 22 22
offset:30
ip:C0 A8 01 0A C0 A8 02 0A

Step 5: Down one or more port of lag and check the lag speed

(1)ovs-ofctl mod-port br0 ge-1/1/2 down

 

ovs-ofctl show br0
check "1025(ae1).*state.* LINK_UP"
check "speed.*200Mbps"

(2)ovs-ofctl mod-port br0 ge-1/1/3 down

 

ovs-ofctl show br0
check "1025(ae1).*state.* LINK_UP"
check "speed.*100Mbps"

(3)ovs-ofctl mod-port br0 ge-1/1/4 down

 

ovs-ofctl show br0
check "1025(ae1).*state.* LINK_DOWN"
check  "speed.*0Mbps"

Step 6: Up all the ports of lag and check the lag speed

ovs-ofctl mod-port br0 ge-1/1/2 up
ovs-ofctl mod-port br0 ge-1/1/3 up
ovs-ofctl mod-port br0 ge-1/1/4 up
ovs-ofctl show br0
check "1025(ae1).*state.* LINK_UP"
check "speed.*300Mbps"

Step 7: Test end and clear the configure

ovs-vsctl del-br br0






Copyright © 2024 Pica8 Inc. All Rights Reserved.