EVPN Multihoming (MH) is a standards based replacement for MLAG to achieve standards based all-active server redundancy in datacenters and enterprise networks. Multihoming eliminates the need to have MLAG and unlike MLAG, does not require the group of devices used to create a multihoming group to have inter-connecting peer links.

EVPN multihoming is based on the concept of forming Ethernet Segments (ES). In the context of BGP EVPN, an Ethernet Segment is a set of links connecting a single end host or server with the EVPN VXLAN fabric. In the Figure below, Server1 is multihomed to the VXLAN fabric through links connecting it to VTEP1 and VTEP2, forming an Ethernet Segment. Similarly, Server2, Server3 and Server4 are connected to the fabric through links forming segment 2, segment 3 and segment 4 respectively.

Figure 1. EVPN Multihoming

Open

Key Benefits             

• Interconnecting switch links between TOR switches in a redundancy group are not required.

• There can be more than two TOR switches in the redundancy group. A maximum of 8 devices can form one redundancy group as opposed to a maximum of two devices in an MLAG group.

• Ease of management as it provides a single BGP-EVPN control plane.

• Allows multi-vendor interoperability because of standards based implementation. 

 Brief Mechanism

As shown in Figure 1, if VTEP1 loses direct connection to Server1, it will signal to VTEP2, VTEP3 and VTEP4 that Server1 is no longer accessible and to change their next-hop group for Server1 if there are alternative routes to this ES. Since VTEP2 also has a route to this ES, VTEP3 and VTEP4 will change their next hop groups for Server1 MAC address to go through VTEP2 instead of VTEP1.

EVPN multihoming uses EVPN Type-1, Type-2 and Type-4 routes to discover these Ethernet Segments and deliver traffic to and from these segments. Each Ethernet Segment is identified by a unique ID called Ethernet Segment ID (ESI). ESI is unique across the entire EVPN VXLAN domain on all VTEPs. To configure ESI, we first need to configure the Ethernet Segment System MAC and a local ES-ID. The local ES system MAC and local ES ID are used to automatically generate the ESI. Different ES System MACs can be configured for different Ethernet Segments but the ES System MAC configured on interfaces connecting to the same server or host must be same. In the sections below we will briefly explore EVPN Type-1Auto Discovery and EVPN Type-4 Ethernet Segment Routes.

EVPN Type-1 Ethernet Auto Discovery (A-D) Route

EVPN Type-1 routes are used for fast convergence and aliasing for multihomed sites or end hosts. Fast convergence is important in networks with large number of hosts. When a link failure occurs, devices can signal to upstream PE devices or other VTEPs in the network to change their next-hop groups for the MAC addresses associated with that particular Ethernet Segment. Aliasing is used for load balancing on multiple outgoing links.

Table 1. EVPN Type-1 Route NLRI

Type-1 routes are only originated for multihomed devices, i.e. these routes are sent only when the Ethernet Segment ID is set to a non-zero value.

EVPN Type-4 Ethernet Segment Route

Type-4 routes are used for electing the Designated Forwarder (DF) in EVPN multihoming scenarios. The DF is responsible for sending broadcast, unknown unicast and multicast (BUM) traffic to multihomed hosts or servers. If a server is multihomed to two upstream devices in the EVPN fabric, only one would be elected as the DF for sending BUM traffic to the multihomed server on this ES.

The EVPN NLRI for Type-4 Ethernet Segment Route consists of the following four fields. For multihomed hosts, the Ethernet Segment Identifier (ESI) must be set to a non-zero value. A value of zero for the ESI indicates a single homed device.

Table 2. EVPN Type-4 NLRI

EVPN multihoming works by utilizing EVPN Type-1, Type-2 and Type-4 routes. Three types of EVPN routes are used to fulfill three distinct EVPN multihoming requirements.

The PICOS EVPN multihoming can be summarized in the following three steps

1. EVPN Type-2 routes are used for remote learning of end hosts. As shown in Figure 1 for example, Server1 is multihomed to VTEP1 and VTEP2. Both VTEP1 and VTEP2 will first locally learn Server1’s MAC/IP address and then advertise it to VTEP3 and VTEP4 using EVPN Type-2 routes (remote learning).

2. With EVPN multihoming enabled on switch ports connecting Server1 to VTEP1 and VTEP2, both devices will configure the same Ethernet Subnet Identifier (ESI) on the two links and advertise this information to all the other VTEPs  using Type-4 routes. This will indicate to remote VTEPs that Server1 is multihomed to the EVPN fabric and is accessible through both VTEP1 and VTEP2.

3. For fast convergence and aliasing, VTEP1 and VTEP2 will advertise Type-1 routes. This type of advertisement is sent in the event of a link failure. If the link connecting VTEP1 and Server1 goes down, VTEP1 will advertise Type-1 route indicating to all the other VTEPs that it has lost connection with Server1 (Ethernet Segment). If VTEP1 was elected the designated forward (DF) for this segment, VTEP2 will then assume the role of DF for Server1 whereas VTEP3 and VTEP4 will adjust their next-hop groups for Server1 accordingly.

Designated Forwarder Election

For BGP EVPN, a Designated Forwarder (DF) is responsible for handling Broadcast, Unknown Unicast and Multicast (BUM) traffic for Ethernet Segments. As shown in Figure 1, Server1 constitutes an Ethernet Segment as it is multihomed to two VTEPs in the EVPN overlay. Only one of the two VTEPs has to be elected a DF to deliver BUM traffic to Server1 from remote VTEPs (VTEP3 and VTEP4). The administrator needs to configure EVPN multihoming preference for the Ethernet Segment. A higher preference value wins the election process and assumes the role of the DF. If the preference on both the VTEPs is same then the VTEP with the smallest IP wins the election. It’s important to note that the interface connecting the Ethernet Segment must be an Aggregate-Ethernet interface.

For every Ethernet Segment there must be only one DF.

The following command can be used to set the ES DF preference.

set interface aggregate-ethernet <interface> evpn mh es-df-pref <preference>

VTEP Uplink Status Tracking

Overlay VTEPs are capable of tracking the state of uplinks connecting the overlay leaf devices with underlay spine switches. This feature is automatically enabled when the user enables the EVPN MH. All the downstream links connecting the MH hosts are put in protocol down state when all the uplinks go down. In Figure 1, all the links connecting the underlay spine switches must go down before the downstream links are put in protocol down state. The links will not be put in protocol down state if there is only one link up and active connecting the overlay devices with the underlay.

EVPN MH Split Horizon

Split-horizon technique is used to stop BUM traffic from forming a loop in the EVPN VXLAN network that employs multihoming. Using split-horizon, BUM traffic from the same Ethernet Segment will be dropped by the peer device. As can been seen in Figure 2, Host1 is multihomed into the EVPN VXLAN fabric and connected to both VTEP1 and VTEP2 forming Ethernet Segment ES1.

In the example below, if Host1 sends some BUM traffic to DF and then it is forwarded to the spine switch and then relayed back to VTEP2, that traffic will be discarded at VTEP2 since VTEP2 has an interface configured for the same Ethernet Segment. Split horizon mechanism ensures BUM traffic can’t loop back to Host1 if Host1 originates this traffic.

Figure 2. EVPN MH Split-Horizon

Open

Key points to note for EVPN MH Split Horizon and BUM traffic duplication:

1. Only one DF is elected in one ESI (one LAG), so partial connection is not allowed for EVPN MH access connection.

2. Non-DF node will drop the BUM traffic from other nodes, including peer of the same ES. BUM traffic from remote VTEPs is dropped by non-DF nodes to avoid duplication.

EVPN MH Basic Configuration

To configure EVPN multihoming on a LAG interface, the user needs to configure an Ethernet Segment identifier (ES-ID) and an Ethernet Segment System MAC. A type-3 ESI value is automatically generated by using these two parameters. The resulting 10-byte ESI value has the following format, where the XXs denote the 6-byte ES System MAC and the YYs denote the 3-byte local ES ID.

03 : XX : XX : XX : XX : XX : XX : YY : YY : YY

For each Ethernet segment, both the ES System MAC and the ES ID must be the same on all the switches forming the EVPN MH redundancy group. However, switches can be configured with different system MACs for different Ethernet segments or share the same system MAC for all Ethernet segments.

Aliasing

The aliasing feature provides remote VTEP devices the ability to load balance Layer-2 unicast traffic through other VTEPs that have configured the same ES (Ethernet Segment). VTEP devices in the EVPN multihoming redundancy group always advertise the communicated information of the same ES to all remote VTEPs via MP-BGP protocol. If one of the VTEP device in an ES, say VTEP1 in the following figure, learns the MAC/ARP information and advertises it to the remote VTEP, say VTEP3. Then, VTEP3 adds not only the IP address of VTEP1 as the next hop, but also adds the IP addresses of other VTEPs in the EVPN multihoming redundancy group as the next hop, to automatically form equivalent paths between different VTEPs.

Figure 3. Aliasing

Open

On the remote VTEP, run the following EVPN MAC show commands to view the next hop. It shows the next hop interface is vplag, including all VTEPs in remote ES.