Introduction

EVPN Multihoming (MH) is a standards based replacement for MLAG to achieve standards based all-active server redundancy in datacenters and enterprise networks. Multihoming eliminates the need to have MLAG and unlike MLAG, does not require the group of devices used to create a multihoming group to have inter-connecting peer links.

EVPN multihoming is based on the concept of forming Ethernet Segments (ES). In the context of BGP EVPN, an Ethernet Segment is a set of links connecting a single end host or server with the EVPN VXLAN fabric. In the Figure below, Server1 is multihomed to the VXLAN fabric through links connecting it to VTEP1 and VTEP2, forming an Ethernet Segment. Similarly, Server2, Server3 and Server4 are connected to the fabric through links forming segment 2, segment 3 and segment 4 respectively.

Figure 1. EVPN Multihoming

Key Benefits

Interconnecting switch links between TOR switches in a redundancy group are not required.
There can be more than two TOR switches in the redundancy group. A maximum of 8 devices can form one redundancy group as opposed to a maximum of two devices in an MLAG group.
Ease of management as it provides a single BGP-EVPN control plane.
Allows multi-vendor interoperability because of standards based implementation.

Brief Mechanism

As shown in Figure 1, if VTEP1 loses direct connection to Server1, it will signal to VTEP2, VTEP3 and VTEP4 that Server1 is no longer accessible and to change their next-hop group for Server1 if there are alternative routes to this ES. Since VTEP2 also has a route to this ES, VTEP3 and VTEP4 will change their next hop groups for Server1 MAC address to go through VTEP2 instead of VTEP1.

EVPN multihoming uses EVPN Type-1, Type-2 and Type-4 routes to discover these Ethernet Segments and deliver traffic to and from these segments. Each Ethernet Segment is identified by a unique ID called Ethernet Segment ID (ESI). ESI is unique across the entire EVPN VXLAN domain on all VTEPs. To configure ESI, we first need to configure the Ethernet Segment System MAC and a local ES-ID. The local ES system MAC and local ES ID are used to automatically generate the ESI. Different ES System MACs can be configured for different Ethernet Segments but the ES System MAC configured on interfaces connecting to the same server or host must be same. In the sections below we will briefly explore EVPN Type-1Auto Discovery and EVPN Type-4 Ethernet Segment Routes.

EVPN Type-1 Ethernet Auto Discovery (A-D) Route

EVPN Type-1 routes are used for fast convergence and aliasing for multihomed sites or end hosts. Fast convergence is important in networks with large number of hosts. When a link failure occurs, devices can signal to upstream PE devices or other VTEPs in the network to change their next-hop groups for the MAC addresses associated with that particular Ethernet Segment. Aliasing is used for load balancing on multiple outgoing links.

Route Distinguisher (RD) (8 octets)

Ethernet Segment Identifier (10 octets)

Ethernet Tag ID (4 octets)

MPLS Label (3 octets)

Table 1. EVPN Type-1 Route NLRI

Type-1 routes are only originated for multihomed devices, i.e. these routes are sent only when the Ethernet Segment ID is set to a non-zero value.

EVPN Type-4 Ethernet Segment Route

Type-4 routes are used for electing the Designated Forwarder (DF) in EVPN multihoming scenarios. The DF is responsible for sending broadcast, unknown unicast and multicast (BUM) traffic to multihomed hosts or servers. If a server is multihomed to two upstream devices in the EVPN fabric, only one would be elected as the DF for sending BUM traffic to the multihomed server on this ES.

The EVPN NLRI for Type-4 Ethernet Segment Route consists of the following four fields. For multihomed hosts, the Ethernet Segment Identifier (ESI) must be set to a non-zero value. A value of zero for the ESI indicates a single homed device.

RD (8 Octets)

Ethernet Segment Identifier (10 Octets)

IP Address Length (1 octet)

Originating Router's IP Address (4 or 16 octets)

Table 2. EVPN Type-4 NLRI

EVPN multihoming works by utilizing EVPN Type-1, Type-2 and Type-4 routes. Three types of EVPN routes are used to fulfill three distinct EVPN multihoming requirements.

The PICOS EVPN multihoming can be summarized in the following three steps

Step1:

EVPN Type-2 routes are used for remote learning of end hosts. As shown in Figure 1 for example, Server1 is multihomed to VTEP1 and VTEP2. Both VTEP1 and VTEP2 will first locally learn Server1’s MAC/IP address and then advertise it to VTEP3 and VTEP4 using EVPN Type-2 routes (remote learning).

Step2:

With EVPN multihoming enabled on switch ports connecting Server1 to VTEP1 and VTEP2, both devices will configure the same Ethernet Subnet Identifier (ESI) on the two links and advertise this information to all the other VTEPs using Type-4 routes. This will indicate to remote VTEPs that Server1 is multihomed to the EVPN fabric and is accessible through both VTEP1 and VTEP2.

Step3:

For fast convergence and aliasing, VTEP1 and VTEP2 will advertise Type-1 routes. This type of advertisement is sent in the event of a link failure. If the link connecting VTEP1 and Server1 goes down, VTEP1 will advertise Type-1 route indicating to all the other VTEPs that it has lost connection with Server1 (Ethernet Segment). If VTEP1 was elected the designated forward (DF) for this segment, VTEP2 will then assume the role of DF for Server1 whereas VTEP3 and VTEP4 will adjust their next-hop groups for Server1 accordingly.

Note

It is strongly suggested to remove any MLAG related configuration from the overlay devices (VTEPs) if you plan on deploying EVPN MH in your network. The two technologies are designed to address almost similar requirements of redundancy but MLAG is a layer2 technique whereas EVPN MH is deployed in the overlay and is a layer3 technique. The two technologies hence do not mix well and users are urged to stick to EVPN MH for its standards based implementation and interoperability benefits as opposed to MLAG especially in the overlay network design.

Designated Forwarder Election

For BGP EVPN, a Designated Forwarder (DF) is responsible for handling Broadcast, Unknown Unicast and Multicast (BUM) traffic for Ethernet Segments. As shown in Figure 1, Server1 constitutes an Ethernet Segment as it is multihomed to two VTEPs in the EVPN overlay. Only one of the two VTEPs has to be elected a DF to deliver BUM traffic to Server1 from remote VTEPs (VTEP3 and VTEP4). The administrator needs to configure EVPN multihoming preference for the Ethernet Segment. A higher preference value wins the election process and assumes the role of the DF. If the preference on both the VTEPs is same then the VTEP with the smallest IP wins the election. It’s important to note that the interface connecting the Ethernet Segment must be an Aggregate-Ethernet interface.

For every Ethernet Segment there must be only one DF.

The following command can be used to set the ES DF preference.

set interface aggregate-ethernet <interface> evpn mh es-df-pref <preference>

VTEP Uplink Status Tracking

Overlay VTEPs are capable of tracking the state of uplinks connecting the overlay leaf devices with underlay spine switches. This feature is automatically enabled when the user enables the EVPN MH. All the downstream links connecting the MH hosts are put in protocol down state when all the uplinks go down. In Figure 1, all the links connecting the underlay spine switches must go down before the downstream links are put in protocol down state. The links will not be put in protocol down state if there is only one link up and active connecting the overlay devices with the underlay.

EVPN MH Split Horizon

Split-horizon technique is used to stop BUM traffic from forming a loop in the EVPN VXLAN network that employs multihoming. Using split-horizon, BUM traffic from the same Ethernet Segment will be dropped by the peer device. As can been seen in Figure 2, Host1 is multihomed into the EVPN VXLAN fabric and connected to both VTEP1 and VTEP2 forming Ethernet Segment ES1.

In the example below, if Host1 sends some BUM traffic to DF and then it is forwarded to the spine switch and then relayed back to VTEP2, that traffic will be discarded at VTEP2 since VTEP2 has an interface configured for the same Ethernet Segment. Split horizon mechanism ensures BUM traffic can’t loop back to Host1 if Host1 originates this traffic.

Figure 2. EVPN MH Split-Horizon

Key points to note for EVPN MH Split Horizon and BUM traffic duplication:

Only one DF is elected in one ESI (one LAG), so partial connection is not allowed for EVPN MH access connection.
Non-DF node will drop the BUM traffic from other nodes, including peer of the same ES. BUM traffic from remote VTEPs is dropped by non-DF nodes to avoid duplication.

Note

Note: Single homed ports connecting a single device with the EVPN VXLAN fabric are not yet supported on VTEP devices. If a server is dual homed to two VTEPs and the connecting link to one of the VTEP fails, the server will still be considered a dual homed device even if there is only one active link connecting it to the fabric.

EVPN MH Basic Configuration

To configure EVPN multihoming on a LAG interface, the user needs to configure an Ethernet Segment identifier (ES-ID) and an Ethernet Segment System MAC. A type-3 ESI value is automatically generated by using these two parameters. The resulting 10-byte ESI value has the following format, where the XXs denote the 6-byte ES System MAC and the YYs denote the 3-byte local ES ID.

03 : XX : XX : XX : XX : XX : XX : YY : YY : YY

For each Ethernet segment, both the ES System MAC and the ES ID must be the same on all the switches forming the EVPN MH redundancy group. However, switches can be configured with different system MACs for different Ethernet segments or share the same system MAC for all Ethernet segments.

Aliasing

The aliasing feature provides remote VTEP devices the ability to load balance Layer-2 unicast traffic through other VTEPs that have configured the same ES (Ethernet Segment). VTEP devices in the EVPN multihoming redundancy group always advertise the communicated information of the same ES to all remote VTEPs via MP-BGP protocol. If one of the VTEP device in an ES, say VTEP1 in the following figure, learns the MAC/ARP information and advertises it to the remote VTEP, say VTEP3. Then, VTEP3 adds not only the IP address of VTEP1 as the next hop, but also adds the IP addresses of other VTEPs in the EVPN multihoming redundancy group as the next hop, to automatically form equivalent paths between different VTEPs.

EVPN multihoming aliasing function uses the ECMP hash fields "ingress-interface","ip-destination", "ip-source," "port-destination," "port-source," and "vlan" for equivalent paths calculation. By default, all these fields are enabled. Users can run the following CLI commands to enable or disable the ECMP hash fields:

set interface ecmp hash-mapping field ingress-interface disable

set interface ecmp hash-mapping field ip-destination disable

set interface ecmp hash-mapping field ip-protocol disable

set interface ecmp hash-mapping field ip-source disable

set interface ecmp hash-mapping field port-destination disable

set interface ecmp hash-mapping field port-source disable

set interface ecmp hash-mapping field vlan disable

On the remote VTEP, run the following EVPN MAC show commands to view the next hop. It shows the next hop interface is vplag, including all VTEPs in remote ES.

admin@Xorplus# run show mac-address table
Total entries in switching table:   1
Static entries in switching table:  0
Dynamic entries in switching table: 1

VLAN         MAC address          Type         Age     Interfaces         User
----         -----------------    ---------    ----    ----------------   ----------  
N/A          00:0c:29:77:8b:15    Dynamic      300     vplag-0x1          xorp


admin@Xorplus# run show vxlan address-table
VNID          MAC address          Type          Interface          VTEP
-----------   -----------------    -------       ----------------   ---------------
10020         00:0c:29:77:8b:15    Dynamic                          10.226.14.253,10.226.14.254
Entries in access port:  0
Entries in network port: 1

Switch Configuration Example and Topology

In PICOS EVPN multihoming, there are two ways to forward BUM messages. You can choose one of these to apply to your EVPN fabric.

Head End Replication. This is the default way which needs no PIM configurations. The BUM message will be encapsulated and sent to all outgoing ports VTEPs of the same VNI by the access VTEP.
BUM tunnel. All VTEPs of the same VNI join the same multicast group, and use multicast routing protocol PIM to establish a multicast forwarding table entry for the multicast group. This method can reduce the traffic flooding caused by the head end replication of the BUM messages. This way needs to configure PIM protocol.

EVPN Multihoming with Head End Replication

For this example configuration we have the following topology comprising of two spine switches, four leaf switches and four servers that are multihomed to this network. Two servers are multihomed to Leaf1 and Leaf2 switches whereas the other two servers are attached to Leaf3 and Leaf4 switches.

The four leaf switches form two MH redundancy groups for the four ES IDs. Two ES ID’s are configured under Leaf1 and Leaf2 whereas, another two ES ID’s are configured under Leaf3 and Leaf4. Each leaf switch has one physical interface added to a LAG interface to connect to the downstream server.

The two ES IDs are configured on each peer connected to the two downstream multihomed servers. On Leaf1, two physical ports connect to the multihomed servers; both the physical ports are first added to LAG interfaces. The LAG interface connected to Server1 is “ae2” whereas the interface connecting Server2 is “ae1”. For EVPN MH, it is a mandatory that the ports connecting the multihomed hosts must be LAG interfaces. Similarly, on all leaf switches there are two LAG interfaces and two ES IDs configured.

To implement a seamless migration of VMs without changing network settings or disrupting the traffic forwarding, configure the same anycast gateway IP and the same virtual MAC (router MAC) on all four leaf switches to enable the anycast gateway function.

The following configuration examples use the topology illustrated below and configure EVPN multihoming with head end replication.

Figure 3. EVPN MH Example Topology

Note

Note: The physical ports connecting the multihomed servers or hosts must be added to a LAG interface and then configured for EVPN MH. Physical interfaces such as ge-1/1/1 cannot be directly used for MH configuration.

Leaf1 Configuration

Step 1. Configure LAG interfaces, set the LAG interface MTU, configure ES ID, configure the LAG interface VLAN membership and the LAG interface trunk mode. MTU configuration is optional, it can be configured when the MTU size needs to be of a particular size based on the customer use case environment. The LACP configuration is also optional.

admin@Leaf1# set interface aggregate-ethernet ae1 aggregated-ether-options lacp enable true
admin@Leaf1# set interface aggregate-ethernet ae2 aggregated-ether-options lacp enable true
admin@Leaf1# set interface aggregate-ethernet ae1 family ethernet-switching port-mode trunk
admin@Leaf1# set interface aggregate-ethernet ae1 family ethernet-switching vlan members 30
admin@Leaf1# set interface aggregate-ethernet ae1 family ethernet-switching vlan members 20
admin@Leaf1# set interface aggregate-ethernet ae2 family ethernet-switching port-mode trunk
admin@Leaf1# set interface aggregate-ethernet ae2 family ethernet-switching vlan members 30
admin@Leaf1# set interface aggregate-ethernet ae2 family ethernet-switching vlan members 20
admin@Leaf1# set interface aggregate-ethernet ae1 evpn mh es-id 4
admin@Leaf1# set interface aggregate-ethernet ae2 evpn mh es-id 2

Step2. Configure ES system MAC, it can be the same or different for different ethernet segments. Configure the physical interfaces and VLAN membership and the ports trunk mode. Also add physical interfaces to LAG interfaces.

admin@Leaf1# set interface aggregate-ethernet ae1 evpn mh es-sys-mac 00:22:22:22:22:22
admin@Leaf1# set interface aggregate-ethernet ae2 evpn mh es-sys-mac 00:22:22:22:22:22
admin@Leaf1# set interface gigabit-ethernet ge-1/1/19 mtu 3000
admin@Leaf1# set interface gigabit-ethernet ge-1/1/19 family ethernet-switching native-vlan-id 4014
admin@Leaf1# set interface gigabit-ethernet ge-1/1/23 ether-options 802.3ad ae2
admin@Leaf1# set interface gigabit-ethernet ge-1/1/23 family ethernet-switching
admin@Leaf1# set interface gigabit-ethernet ge-1/1/24 disable true
admin@Leaf1# set interface gigabit-ethernet ge-1/1/27 mtu 3000
admin@Leaf1# set interface gigabit-ethernet ge-1/1/27 family ethernet-switching native-vlan-id 4012
admin@Leaf1# set interface gigabit-ethernet ge-1/1/37 ether-options 802.3ad ae1

Step3. Configure VRF and enable IP routing.

admin@Leaf1# set ip routing enable true
admin@Leaf1# set ip vrf vrf1

Step4. Configure loopback interfaces and assign IP addresses.

admin@Leaf1# set l3-interface loopback lo address 10.226.14.254 prefix-length 32
admin@Leaf1# set l3-interface loopback lo address 10.226.14.207 prefix-length 32
admin@Leaf1# set l3-interface loopback vrf1 address 17.17.17.17 prefix-length 32

Step5. Configure VLAN interfaces and assign IP addresses, configure anycast gateway.

admin@Leaf1# set l3-interface vlan-interface vlan4014 address 10.226.15.65 prefix-length 30
admin@Leaf1# set l3-interface vlan-interface vlan4012 address 10.226.15.29 prefix-length 30
admin@Leaf1# set l3-interface vlan-interface vlan4094 address 10.226.13.26 prefix-length 30
admin@Leaf1# set l3-interface vlan-interface vlan99 vrf vrf1
admin@Leaf1# set l3-interface vlan-interface vlan20 vrf vrf1
admin@Leaf1# set l3-interface vlan-interface vlan20 address 192.168.0.1 prefix-length 24
admin@Leaf1# set l3-interface vlan-interface vlan20 address 2002:0:0:1::1 prefix-length 64
admin@Leaf1# set l3-interface vlan-interface vlan30 vrf vrf1
admin@Leaf1# set l3-interface vlan-interface vlan30 address 172.168.0.1 prefix-length 24
admin@Leaf1# set l3-interface vlan-interface vlan30 address 2003:0:0:1::1 prefix-length 64
admin@Leaf1# set l3-interface vlan-interface vlan20 anycast address 192.168.0.20 prefix-length 24
admin@Leaf1# set l3-interface vlan-interface vlan20 anycast address 2002:0:0:1::20 prefix-length 64
admin@Leaf1# set l3-interface vlan-interface vlan30 anycast address 172.168.0.30 prefix-length 24
admin@Leaf1# set l3-interface vlan-interface vlan30 anycast address 2003:0:0:1::30 prefix-length 64 
admin@Leaf1# set l3-interface vlan-interface vlan20 anycast mac 00:00:20:00:00:FE
admin@Leaf1# set l3-interface vlan-interface vlan30 anycast mac 00:00:30:00:00:FE

Step6. Configure BGP and EVPN.

admin@Leaf1# set protocols bgp local-as 65001
admin@Leaf1# set protocols bgp router-id 10.226.14.207
admin@Leaf1# set protocols bgp neighbor 10.226.14.48 remote-as internal
admin@Leaf1# set protocols bgp neighbor 10.226.14.48 update-source 10.226.14.207
admin@Leaf1# set protocols bgp neighbor 10.226.14.48 evpn activate true
admin@Leaf1# set protocols bgp neighbor 10.226.14.208 remote-as internal
admin@Leaf1# set protocols bgp neighbor 10.226.14.208 update-source 10.226.14.207
admin@Leaf1# set protocols bgp neighbor 10.226.14.208 evpn activate true
admin@Leaf1# set protocols bgp neighbor 10.226.14.24 remote-as internal
admin@Leaf1# set protocols bgp neighbor 10.226.14.24 update-source 10.226.14.207
admin@Leaf1# set protocols bgp neighbor 10.226.14.24 evpn activate true
admin@Leaf1# set protocols bgp ipv4-unicast
admin@Leaf1# set protocols bgp evpn advertise-all-vni
admin@Leaf1# set protocols bgp evpn advertise ipv4-unicast
admin@Leaf1# set protocols bgp evpn advertise ipv6-unicast
admin@Leaf1# set protocols bgp evpn advertise-svi-ip
admin@Leaf1# set protocols bgp vrf vrf1 local-as 65001
admin@Leaf1# set protocols bgp vrf vrf1 router-id 17.17.17.17
admin@Leaf1# set protocols bgp vrf vrf1 evpn advertise ipv4-unicast
admin@Leaf1# set protocols bgp vrf vrf1 evpn advertise ipv6-unicast
admin@Leaf1# set protocols evpn mh
admin@Leaf1# set protocols lldp enable true

Step7. Configure OSPF.

admin@Leaf1# set protocols ospf router-id 17.17.17.17
admin@Leaf1# set protocols ospf network 10.226.15.64/30 area 0.0.0.0
admin@Leaf1# set protocols ospf network 10.226.15.28/30 area 0.0.0.0
admin@Leaf1# set protocols ospf network 10.226.14.254/32 area 0.0.0.0
admin@Leaf1# set protocols ospf network 10.226.14.207/32 area 0.0.0.0

Step8. Configure hostname, inband mode and VLANs.

admin@Leaf1# set system hostname Leaf1
admin@Leaf1# set system inband enable true
admin@Leaf1# set vlans vlan-id 20 l3-interface vlan20
admin@Leaf1# set vlans vlan-id 30 l3-interface vlan30
admin@Leaf1# set vlans vlan-id 99 l3-interface vlan99
admin@Leaf1# set vlans vlan-id 4012 l3-interface vlan4012
admin@Leaf1# set vlans vlan-id 4014 l3-interface vlan4014
admin@Leaf1# set vlans vlan-id 4094 l3-interface vlan4094

Step9. Configure VXLANs.

admin@Leaf1# set vxlans source-interface lo address 10.226.14.254
admin@Leaf1# set vxlans vni 10030 decapsulation mode service-vlan-per-port
admin@Leaf1# set vxlans vni 10030 vlan 30
admin@Leaf1# set vxlans vni 10030 arp-nd-suppress disable false
admin@Leaf1# set vxlans vni 10020 decapsulation mode service-vlan-per-port
admin@Leaf1# set vxlans vni 10020 vlan 20
admin@Leaf1# set vxlans vni 10020 arp-nd-suppress disable false
admin@Leaf1# set vxlans vni 99 vlan 99
admin@Leaf1# set vxlans vrf vrf1 l3-vni 99
admin@Leaf1# commit

Leaf2 Configuration

admin@Leaf2# set interface aggregate-ethernet ae1 aggregated-ether-options lacp enable true
admin@Leaf2# set interface aggregate-ethernet ae2 aggregated-ether-options lacp enable true
admin@Leaf2# set interface aggregate-ethernet ae1 family ethernet-switching port-mode trunk
admin@Leaf2# set interface aggregate-ethernet ae1 family ethernet-switching vlan members 30
admin@Leaf2# set interface aggregate-ethernet ae1 family ethernet-switching vlan members 20
admin@Leaf2# set interface aggregate-ethernet ae1 evpn mh es-id 4
admin@Leaf2# set interface aggregate-ethernet ae1 evpn mh es-sys-mac 00:22:22:22:22:22
admin@Leaf2# set interface aggregate-ethernet ae2 family ethernet-switching port-mode "trunk"
admin@Leaf2# set interface aggregate-ethernet ae2 family ethernet-switching vlan members 30
admin@Leaf2# set interface aggregate-ethernet ae2 family ethernet-switching vlan members 20
admin@Leaf2# set interface aggregate-ethernet ae2 evpn mh es-id 2
admin@Leaf2# set interface aggregate-ethernet ae2 evpn mh es-sys-mac 00:22:22:22:22:22
admin@Leaf2# set interface gigabit-ethernet te-1/1/17 mtu 3000
admin@Leaf2# set interface gigabit-ethernet te-1/1/17 family ethernet-switching native-vlan-id 4014
admin@Leaf2# set interface gigabit-ethernet te-1/1/19 mtu 3000
admin@Leaf2# set interface gigabit-ethernet te-1/1/19 family ethernet-switching native-vlan-id 4013
admin@Leaf2# set interface gigabit-ethernet te-1/1/21 ether-options 802.3ad ae2
admin@Leaf2# set interface gigabit-ethernet te-1/1/23 ether-options 802.3ad ae1
admin@Leaf2# set interface ethernet-switching-options
admin@Leaf2# set ip routing enable true
admin@Leaf2# set ip vrf vrf1
admin@Leaf2# set l3-interface loopback lo address 10.226.14.253 prefix-length 32
admin@Leaf2# set l3-interface loopback vrf1 address 26.26.26.26 prefix-length 32
admin@Leaf2# set l3-interface vlan-interface vlan4013 address 10.226.15.25 prefix-length 30
admin@Leaf2# set l3-interface vlan-interface vlan4014 address 10.226.15.81 prefix-length 30
admin@Leaf2# set l3-interface vlan-interface vlan4094 address 10.226.13.25 prefix-length 30
admin@Leaf2# set l3-interface vlan-interface vlan99 vrf vrf1
admin@Leaf2# set l3-interface vlan-interface vlan20 vrf vrf1
admin@Leaf2# set l3-interface vlan-interface vlan20 address 192.168.0.2 prefix-length 24
admin@Leaf2# set l3-interface vlan-interface vlan20 address 2002:0:0:1::2 prefix-length 64
admin@Leaf2# set l3-interface vlan-interface vlan30 vrf vrf1
admin@Leaf2# set l3-interface vlan-interface vlan30 address 172.168.0.2 prefix-length 24
admin@Leaf2# set l3-interface vlan-interface vlan30 address 2003:0:0:1::2 prefix-length 64
admin@Leaf2# set l3-interface vlan-interface vlan20 anycast address 192.168.0.20 prefix-length 24
admin@Leaf2# set l3-interface vlan-interface vlan20 anycast address 2002:0:0:1::20 prefix-length 64
admin@Leaf2# set l3-interface vlan-interface vlan30 anycast address 172.168.0.30 prefix-length 24
admin@Leaf2# set l3-interface vlan-interface vlan30 anycast address 2003:0:0:1::30 prefix-length 64  
admin@Leaf2# set l3-interface vlan-interface vlan20 anycast mac 00:00:20:00:00:FE
admin@Leaf2# set l3-interface vlan-interface vlan30 anycast mac 00:00:30:00:00:FE
admin@Leaf2# set protocols bgp local-as 65001
admin@Leaf2# set protocols bgp router-id 10.226.14.208
admin@Leaf2# set protocols bgp neighbor 10.226.14.48 remote-as internal
admin@Leaf2# set protocols bgp neighbor 10.226.14.48 update-source 10.226.14.208
admin@Leaf2# set protocols bgp neighbor 10.226.14.48 evpn activate true
admin@Leaf2# set protocols bgp neighbor 10.226.14.207 remote-as internal
admin@Leaf2# set protocols bgp neighbor 10.226.14.207 update-source 10.226.14.208
admin@Leaf2# set protocols bgp neighbor 10.226.14.207 evpn activate true
admin@Leaf2# set protocols bgp neighbor 10.226.14.24 remote-as internal
admin@Leaf2# set protocols bgp neighbor 10.226.14.24 update-source 10.226.14.208
admin@Leaf2# set protocols bgp neighbor 10.226.14.24 evpn activate true
admin@Leaf2# set protocols bgp ipv4-unicast
admin@Leaf2# set protocols bgp evpn advertise-all-vni
admin@Leaf2# set protocols bgp evpn advertise ipv4-unicast
admin@Leaf2# set protocols bgp evpn advertise ipv6-unicast
admin@Leaf2# set protocols bgp evpn advertise-svi-ip
admin@Leaf2# set protocols bgp vrf vrf1 local-as 65001
admin@Leaf2# set protocols bgp vrf vrf1 router-id 26.26.26.26
admin@Leaf2# set protocols bgp vrf vrf1 evpn advertise ipv4-unicast
admin@Leaf2# set protocols bgp vrf vrf1 evpn advertise ipv6-unicast
admin@Leaf2# set protocols evpn mh
admin@Leaf2# set protocols lldp enable true
admin@Leaf2# set protocols ospf router-id 110.110.110.110
admin@Leaf2# set protocols ospf network 10.226.15.24/30 area 0.0.0.0
admin@Leaf2# set protocols ospf network 10.226.15.80/30 area 0.0.0.0
admin@Leaf2# set protocols ospf network 10.226.14.208/32 area 0.0.0.0
admin@Leaf2# set protocols ospf network 10.226.14.253/32 area 0.0.0.0
admin@Leaf2# set vlans vlan-id 20 l3-interface vlan20
admin@Leaf2# set vlans vlan-id 30 l3-interface vlan30
admin@Leaf2# set vlans vlan-id 99 l3-interface vlan99
admin@Leaf2# set vlans vlan-id 4013 l3-interface vlan4013
admin@Leaf2# set vlans vlan-id 4014 l3-interface vlan4014
admin@Leaf2# set vlans vlan-id 4094 l3-interface vlan4094
admin@Leaf2# set vxlans source-interface lo address 10.226.14.253
admin@Leaf2# set vxlans vni 10030 decapsulation mode service-vlan-per-port
admin@Leaf2# set vxlans vni 10030 vlan 30
admin@Leaf2# set vxlans vni 10030 arp-nd-suppress disable false
admin@Leaf2# set vxlans vni 10020 decapsulation mode service-vlan-per-port
admin@Leaf2# set vxlans vni 10020 vlan 20
admin@Leaf2# set vxlans vni 10020 arp-nd-suppress disable false
admin@Leaf2# set vxlans vni 99 vlan 99
admin@Leaf2# set vxlans vrf vrf1 l3-vni 99
admin@Leaf2# commit

Leaf3 Configuration

admin@Leaf3# set interface aggregate-ethernet ae1 aggregated-ether-options lacp enable true
admin@Leaf3# set interface aggregate-ethernet ae2 aggregated-ether-options lacp enable true
admin@Leaf3# set interface aggregate-ethernet ae1 family ethernet-switching port-mode trunk
admin@Leaf3# set interface aggregate-ethernet ae1 family ethernet-switching vlan members 30
admin@Leaf3# set interface aggregate-ethernet ae1 family ethernet-switching vlan members 20
admin@Leaf3# set interface aggregate-ethernet ae1 evpn mh es-id 3
admin@Leaf3# set interface aggregate-ethernet ae1 evpn mh es-sys-mac 00:00:00:22:22:22
admin@Leaf3# set interface aggregate-ethernet ae2 family ethernet-switching port-mode trunk
admin@Leaf3# set interface aggregate-ethernet ae2 family ethernet-switching vlan members 30
admin@Leaf3# set interface aggregate-ethernet ae2 family ethernet-switching vlan members 20
admin@Leaf3# set interface aggregate-ethernet ae2 evpn mh es-id 1
admin@Leaf3# set interface aggregate-ethernet ae2 evpn mh es-sys-mac 00:00:00:22:22:22
admin@Leaf3# set interface gigabit-ethernet ge-1/1/3 mtu 3000
admin@Leaf3# set interface gigabit-ethernet ge-1/1/3 family ethernet-switching native-vlan-id 3011
admin@Leaf3# set interface gigabit-ethernet ge-1/1/5 mtu 3000
admin@Leaf3# set interface gigabit-ethernet ge-1/1/5 family ethernet-switching native-vlan-id 3012
admin@Leaf3# set interface gigabit-ethernet ge-1/1/21 ether-options 802.3ad ae1
admin@Leaf3# set interface gigabit-ethernet ge-1/1/22 ether-options 802.3ad ae2
admin@Leaf3# set interface ethernet-switching-options
admin@Leaf3# set ip routing enable true
admin@Leaf3# set ip vrf vrf1
admin@Leaf3# set l3-interface loopback lo address 10.226.14.24 prefix-length 32
admin@Leaf3# set l3-interface loopback lo address 10.226.14.201 prefix-length 32
admin@Leaf3# set l3-interface vlan-interface vlan3011 address 10.226.0.98 prefix-length 30
admin@Leaf3# set l3-interface vlan-interface vlan3012 address 10.226.0.94 prefix-length 30
admin@Leaf3# set l3-interface vlan-interface vlan99 vrf vrf1
admin@Leaf3# set l3-interface vlan-interface vlan20 vrf vrf1
admin@Leaf3# set l3-interface vlan-interface vlan20 address 192.168.0.3 prefix-length 24
admin@Leaf3# set l3-interface vlan-interface vlan20 address 2002:0:0:1::3 prefix-length 64
admin@Leaf3# set l3-interface vlan-interface vlan30 vrf vrf1
admin@Leaf3# set l3-interface vlan-interface vlan30 address 172.168.0.3 prefix-length 24
admin@Leaf3# set l3-interface vlan-interface vlan30 address 2003:0:0:1::3 prefix-length 64
admin@Leaf3# set l3-interface vlan-interface vlan20 anycast address 192.168.0.20 prefix-length 24
admin@Leaf3# set l3-interface vlan-interface vlan20 anycast address 2002:0:0:1::20 prefix-length 64
admin@Leaf3# set l3-interface vlan-interface vlan30 anycast address 172.168.0.30 prefix-length 24
admin@Leaf3# set l3-interface vlan-interface vlan30 anycast address 2003:0:0:1::30 prefix-length 64  
admin@Leaf3# set l3-interface vlan-interface vlan20 anycast mac 00:00:20:00:00:FE
admin@Leaf3# set l3-interface vlan-interface vlan30 anycast mac 00:00:30:00:00:FE
admin@Leaf3# set protocols bgp local-as 65001
admin@Leaf3# set protocols bgp router-id 10.226.14.24
admin@Leaf3# set protocols bgp neighbor 10.226.14.207 remote-as internal
admin@Leaf3# set protocols bgp neighbor 10.226.14.207 timers delayopen 60
admin@Leaf3# set protocols bgp neighbor 10.226.14.207 update-source "10.226.14.24"
admin@Leaf3# set protocols bgp neighbor 10.226.14.207 evpn activate true
admin@Leaf3# set protocols bgp neighbor 10.226.14.208 remote-as "internal"
admin@Leaf3# set protocols bgp neighbor 10.226.14.208 timers delayopen 60
admin@Leaf3# set protocols bgp neighbor 10.226.14.208 update-source 10.226.14.24
admin@Leaf3# set protocols bgp neighbor 10.226.14.208 evpn activate true
admin@Leaf3# set protocols bgp neighbor 10.226.14.48 remote-as internal
admin@Leaf3# set protocols bgp neighbor 10.226.14.48 timers delayopen 60
admin@Leaf3# set protocols bgp neighbor 10.226.14.48 update-source 10.226.14.24
admin@Leaf3# set protocols bgp neighbor 10.226.14.48 evpn activate true
admin@Leaf3# set protocols bgp ipv4-unicast
admin@Leaf3# set protocols bgp evpn advertise-all-vni
admin@Leaf3# set protocols bgp evpn advertise ipv4-unicast
admin@Leaf3# set protocols bgp evpn advertise ipv6-unicast
admin@Leaf3# set protocols bgp evpn advertise-svi-ip
admin@Leaf3# set protocols lldp enable true
admin@Leaf3# set protocols ospf router-id 24.24.24.24
admin@Leaf3# set protocols ospf network 10.226.0.96/30 area 0.0.0.0
admin@Leaf3# set protocols ospf network 10.226.0.92/30 area 0.0.0.0
admin@Leaf3# set protocols ospf network 10.226.14.201/32 area 0.0.0.0
admin@Leaf3# set protocols ospf network 10.226.14.24/32 area 0.0.0.0
admin@Leaf3# set vlans vlan-id 20 l3-interface vlan20
admin@Leaf3# set vlans vlan-id 30 l3-interface vlan30
admin@Leaf3# set vlans vlan-id 99 l3-interface vlan99
admin@Leaf3# set vlans vlan-id 3012 l3-interface vlan3012
admin@Leaf3# set vlans vlan-id 3011 l3-interface vlan3011
admin@Leaf3# set vxlans source-interface lo address 10.226.14.201
admin@Leaf3# set vxlans vni 10020 decapsulation mode service-vlan-per-port
admin@Leaf3# set vxlans vni 10020 vlan 20
admin@Leaf3# set vxlans vni 10020 arp-nd-suppress disable false
admin@Leaf3# set vxlans vni 10030 decapsulation mode service-vlan-per-port
admin@Leaf3# set vxlans vni 10030 vlan 30
admin@Leaf3# set vxlans vni 10030 arp-nd-suppress disable false
admin@Leaf3# set vxlans vni 99 vlan 99
admin@Leaf3# set vxlans traceoptions flag all disable false
admin@Leaf3# set vxlans vrf vrf1 l3-vni 99
admin@Leaf3# commit

Leaf4 Configuration

admin@Leaf4# set interface aggregate-ethernet ae1 aggregated-ether-options lacp enable true
admin@Leaf4# set interface aggregate-ethernet ae2 aggregated-ether-options lacp enable true
admin@Leaf4# set interface aggregate-ethernet ae1 family ethernet-switching port-mode trunk
admin@Leaf4# set interface aggregate-ethernet ae1 family ethernet-switching vlan members 30
admin@Leaf4# set interface aggregate-ethernet ae1 family ethernet-switching vlan members 20
admin@Leaf4# set interface aggregate-ethernet ae1 evpn mh es-id 1
admin@Leaf4# set interface aggregate-ethernet ae1 evpn mh es-sys-mac 00:00:00:22:22:22
admin@Leaf4# set interface aggregate-ethernet ae2 mtu 3000
admin@Leaf4# set interface aggregate-ethernet ae2 family ethernet-switching port-mode trunk
admin@Leaf4# set interface aggregate-ethernet ae2 family ethernet-switching vlan members 30
admin@Leaf4# set interface aggregate-ethernet ae2 family ethernet-switching vlan members 20
admin@Leaf4# set interface aggregate-ethernet ae2 evpn mh es-id 3
admin@Leaf4# set interface aggregate-ethernet ae2 evpn mh es-sys-mac 00:00:00:22:22:22
admin@Leaf4# set interface gigabit-ethernet ge-1/1/3 mtu 3000
admin@Leaf4# set interface gigabit-ethernet ge-1/1/3 family ethernet-switching native-vlan-id 3011
admin@Leaf4# set interface gigabit-ethernet ge-1/1/5 mtu 3000
admin@Leaf4# set interface gigabit-ethernet ge-1/1/5 family ethernet-switching native-vlan-id 3012
admin@Leaf4# set interface gigabit-ethernet ge-1/1/21 ether-options 802.3ad ae1
admin@Leaf4# set interface gigabit-ethernet ge-1/1/22 ether-options 802.3ad ae2
admin@Leaf4# set ip routing enable true
admin@Leaf4# set ip vrf vrf1
admin@Leaf4# set l3-interface loopback lo address 10.226.14.202 prefix-length 32
admin@Leaf4# set l3-interface loopback lo address 10.226.14.48 prefix-length 32
admin@Leaf4# set l3-interface loopback vrf1 address 48.48.48.48 prefix-length 32
admin@Leaf4# set l3-interface vlan-interface vlan3011 address 10.226.15.70 prefix-length 30
admin@Leaf4# set l3-interface vlan-interface vlan3012 address 10.226.15.78 prefix-length 30
admin@Leaf4# set l3-interface vlan-interface vlan99 vrf vrf1
admin@Leaf4# set l3-interface vlan-interface vlan20 vrf vrf1
admin@Leaf4# set l3-interface vlan-interface vlan20 address 192.168.0.4 prefix-length 24
admin@Leaf4# set l3-interface vlan-interface vlan20 address 2002:0:0:1::4 prefix-length 64
admin@Leaf4# set l3-interface vlan-interface vlan30 vrf vrf1
admin@Leaf4# set l3-interface vlan-interface vlan30 address 172.168.0.4 prefix-length 24
admin@Leaf4# set l3-interface vlan-interface vlan30 address 2003:0:0:1::4 prefix-length 64
admin@Leaf4# set l3-interface vlan-interface vlan20 anycast address 192.168.0.20 prefix-length 24
admin@Leaf4# set l3-interface vlan-interface vlan20 anycast address 2002:0:0:1::20 prefix-length 64
admin@Leaf4# set l3-interface vlan-interface vlan30 anycast address 172.168.0.30 prefix-length 24
admin@Leaf4# set l3-interface vlan-interface vlan30 anycast address 2003:0:0:1::30 prefix-length 64  
admin@Leaf4# set l3-interface vlan-interface vlan20 anycast mac 00:00:20:00:00:FE
admin@Leaf4# set l3-interface vlan-interface vlan30 anycast mac 00:00:30:00:00:FE
admin@Leaf4# set protocols bgp local-as 65001
admin@Leaf4# set protocols bgp router-id 10.226.14.48
admin@Leaf4# set protocols bgp neighbor 10.226.14.207 remote-as internal
admin@Leaf4# set protocols bgp neighbor 10.226.14.207 update-source 10.226.14.48
admin@Leaf4# set protocols bgp neighbor 10.226.14.207 evpn activate true
admin@Leaf4# set protocols bgp neighbor 10.226.14.208 remote-as internal
admin@Leaf4# set protocols bgp neighbor 10.226.14.208 update-source 10.226.14.48
admin@Leaf4# set protocols bgp neighbor 10.226.14.208 evpn activate true
admin@Leaf4# set protocols bgp neighbor 10.226.14.24 remote-as internal
admin@Leaf4# set protocols bgp neighbor 10.226.14.24 update-source 10.226.14.48
admin@Leaf4# set protocols bgp neighbor 10.226.14.24 evpn activate true
admin@Leaf4# set protocols bgp ipv4-unicast
admin@Leaf4# set protocols bgp evpn advertise-all-vni
admin@Leaf4# set protocols bgp evpn advertise ipv4-unicast
admin@Leaf4# set protocols bgp evpn advertise ipv6-unicast
admin@Leaf4# set protocols bgp evpn advertise-svi-ip
admin@Leaf4# set protocols bgp vrf vrf1 local-as 65001
admin@Leaf4# set protocols bgp vrf vrf1 router-id 48.48.48.48
admin@Leaf4# set protocols bgp vrf vrf1 evpn advertise ipv4-unicast
admin@Leaf4# set protocols bgp vrf vrf1 evpn advertise ipv6-unicast
admin@Leaf4# set protocols evpn mh startup-delay 60
admin@Leaf4# set protocols lldp enable true
admin@Leaf4# set protocols ospf router-id 48.48.48.48
admin@Leaf4# set protocols ospf network 10.226.14.202/32 area 0.0.0.0
admin@Leaf4# set protocols ospf network 10.226.14.48/32 area 0.0.0.0
admin@Leaf4# set protocols ospf network 10.226.15.68/30 area 0.0.0.0
admin@Leaf4# set protocols ospf network 10.226.15.76/30 area 0.0.0.0
admin@Leaf4# set vlans vlan-id 20 l3-interface vlan20
admin@Leaf4# set vlans vlan-id 30 l3-interface vlan30
admin@Leaf4# set vlans vlan-id 99 l3-interface vlan99
admin@Leaf4# set vlans vlan-id 3011 l3-interface vlan3011
admin@Leaf4# set vlans vlan-id 3012 l3-interface vlan3012
admin@Leaf4# set vxlans source-interface lo address 10.226.14.202
admin@Leaf4# set vxlans vni 10020 decapsulation mode service-vlan-per-port
admin@Leaf4# set vxlans vni 10020 vlan 20
admin@Leaf4# set vxlans vni 10020 arp-nd-suppress disable false
admin@Leaf4# set vxlans vni 10030 decapsulation mode service-vlan-per-port
admin@Leaf4# set vxlans vni 10030 vlan 30
admin@Leaf4# set vxlans vni 10030 arp-nd-suppress disable false
admin@Leaf4# set vxlans vni 99 vlan 99
admin@Leaf4# set vxlans vrf vrf1 l3-vni 99
admin@Leaf4# commit

Spine1 Configuration

admin@Spine1# set interface gigabit-ethernet te-1/1/35 mtu 2000
admin@Spine1# set interface gigabit-ethernet te-1/1/35 family ethernet-switching native-vlan-id 4013
admin@Spine1# set interface gigabit-ethernet te-1/1/41 mtu 2000
admin@Spine1# set interface gigabit-ethernet te-1/1/41 family ethernet-switching native-vlan-id 4014
admin@Spine1# set interface gigabit-ethernet te-1/1/44 mtu 2000
admin@Spine1# set interface gigabit-ethernet te-1/1/44 family ethernet-switching native-vlan-id 4016
admin@Spine1# set interface gigabit-ethernet te-1/1/45 family ethernet-switching native-vlan-id 4015
admin@Spine1# set ip routing enable true
admin@Spine1# set l3-interface loopback lo address 185.185.185.185 prefix-length 32
admin@Spine1# set l3-interface vlan-interface vlan4014 address 10.226.15.66 prefix-length 30
admin@Spine1# set l3-interface vlan-interface vlan4013 address 10.226.15.26 prefix-length 30
admin@Spine1# set l3-interface vlan-interface vlan4015 address 10.226.15.69 prefix-length 30
admin@Spine1# set l3-interface vlan-interface vlan4016 address 10.226.0.97 prefix-length 30
admin@Spine1# set protocols spanning-tree enable false
admin@Spine1# set protocols ospf router-id 185.185.185.185
admin@Spine1# set protocols ospf network 10.226.15.64/30 area 0.0.0.0
admin@Spine1# set protocols ospf network 10.226.15.24/30 area 0.0.0.0
admin@Spine1# set protocols ospf network 10.226.15.68/30 area 0.0.0.0
admin@Spine1# set protocols ospf network 10.226.0.96/30 area 0.0.0.0
admin@Spine1# set protocols ospf network 185.185.185.185/32 area 0.0.0.0
admin@Spine1# set vlans vlan-id 4013 l3-interface vlan4013
admin@Spine1# set vlans vlan-id 4014 l3-interface vlan4014
admin@Spine1# set vlans vlan-id 4015 l3-interface vlan4015
admin@Spine1# set vlans vlan-id 4016 l3-interface vlan4016
admin@Spine1# commit

Spine2 Configuration

admin@Spine2# set interface gigabit-ethernet te-1/1/27 mtu 2000
admin@Spine2# set interface gigabit-ethernet te-1/1/27 family ethernet-switching native-vlan-id 4012
admin@Spine2# set interface gigabit-ethernet te-1/1/27 speed 1000
admin@Spine2# set interface gigabit-ethernet te-1/1/37 mtu 2000
admin@Spine2# set interface gigabit-ethernet te-1/1/37 family ethernet-switching native-vlan-id 4014
admin@Spine2# set interface gigabit-ethernet te-1/1/37 speed 1000
admin@Spine2# set interface gigabit-ethernet te-1/1/44 speed 1000
admin@Spine2# set interface gigabit-ethernet te-1/1/44 family ethernet-switching native-vlan-id 4015
admin@Spine2# set interface gigabit-ethernet te-1/1/45 family ethernet-switching native-vlan-id 3012
admin@Spine2# set interface gigabit-ethernet te-1/1/45 speed 1000
admin@Spine2# set ip routing enable true
admin@Spine2# set l3-interface loopback lo address 202.202.202.202 prefix-length 32
admin@Spine2# set l3-interface vlan-interface vlan4012 address 10.226.15.30 prefix-length 30
admin@Spine2# set l3-interface vlan-interface vlan4014 address 10.226.15.82 prefix-length 30
admin@Spine2# set l3-interface vlan-interface vlan4015 address 10.226.0.93 prefix-length 30
admin@Spine2# set l3-interface vlan-interface vlan3012 address 10.226.15.77 prefix-length 30 
admin@Spine2# set protocols lldp enable true
admin@Spine2# set protocols spanning-tree enable false
admin@Spine2# set protocols ospf router-id 202.202.202.202
admin@Spine2# set protocols ospf network 10.226.0.92/30 area 0.0.0.0
admin@Spine2# set protocols ospf network 10.226.15.28/30 area 0.0.0.0
admin@Spine2# set protocols ospf network 10.226.15.80/30 area 0.0.0.0
admin@Spine2# set protocols ospf network 10.226.15.76/30 area 0.0.0.0
admin@Spine2# set protocols ospf network 202.202.202.202/32 area 0.0.0.0
admin@Spine2# set vlans vlan-id 4012 l3-interface vlan4012
admin@Spine2# set vlans vlan-id 4014 l3-interface vlan4014
admin@Spine2# set vlans vlan-id 4015 l3-interface vlan4015
admin@Spine2# set vlans vlan-id 3012 l3-interface vlan3012 
admin@Spine2# commit

Verify Configuration

The above configuration uses the same default ES-DF preferences on all the three Leaf switches. The resulting effect is that the leaf switch with the smallest VTEP IP is selected as the DF. In the first EVPN MH redundancy group, Leaf2 is elected as the DF since it has the smallest VTEP IP in the group. Similarly in the second redundancy group, Leaf3 has the smallest VTEP IP and is elected the DF.

admin@Leaf1# run show evpn es 
Type: B bypass, L local, R remote, N non-DF
ESI                               Type   ES-IF           VTEPs
03:00:22:22:22:22:22:00:00:04     LRN   ae1           10.226.14.253
03:00:22:22:22:22:22:00:00:02     LRN   ae2           10.226.14.253

admin@Leaf2# run show evpn es 
Type: B bypass, L local, R remote, N non-DF
ESI                               Type   ES-IF            VTEPs
03:00:22:22:22:22:22:00:00:04     LR    ae1            10.226.14.254
03:00:22:22:22:22:22:00:00:02     LR    ae2            10.226.14.254

admin@Leaf3# run show evpn es
Type: B bypass, L local, R remote, N non-DF
ESI                                Type   ES-IF          VTEPs
03:00:00:00:22:22:22:00:00:03      LR    ae1           10.226.14.202
03:00:00:00:22:22:22:00:00:01      LR    ae2           10.226.14.202

admin@Leaf4# run show evpn es
Type: B bypass, L local, R remote, N non-DF
ESI                                Type   ES-IF           VTEPs
03:00:00:00:22:22:22:00:00:01      LRN    ae1          10.226.14.201
03:00:00:00:22:22:22:00:00:03      LRN    ae2          10.226.14.201

Now let’s change the ES-DF preference on Leaf1 to 65535 and see the output again. This time Leaf1 assumes the role of DF.

admin@Leaf1# set interface aggregate-ethernet ae1 evpn mh es-df-pref 65535
admin@Leaf1# set interface aggregate-ethernet ae2 evpn mh es-df-pref 65535
admin@Leaf1# commit 
Commit OK.
Save done

admin@Leaf1# run show evpn es 
Type: B bypass, L local, R remote, N non-DF
ESI                           Type   ES-IF              VTEPs
03:00:22:22:22:22:22:00:00:04     LR    ae1              10.226.14.253
03:00:22:22:22:22:22:00:00:02     LR    ae2              10.226.14.253

admin@Leaf2# run show evpn es 
Type: B bypass, L local, R remote, N non-DF 
ESI                           Type   ES-IF              VTEPs
03:00:22:22:22:22:22:00:00:04     LRN    ae1            10.226.14.254
03:00:22:22:22:22:22:00:00:02     LRN    ae2            10.226.14.254

To check the EVPN ES detail information, run the following show command. This command shows important information such as ESI, Type, interface, state, DF status and the VTEPs etc.

admin@Leaf3# run show evpn es detail 
ESI: 03:00:00:00:22:22:22:00:00:03
 Type: Local,Remote
 Interface: ae1
 State: up
 Bridge port: yes
 Ready for BGP: yes
 VNI Count: 2
 MAC Count: 0
 DF status: df 
 DF preference: 32767
 Nexthop group: 536870913
 VTEPs:
     10.226.14.202 df_alg: preference df_pref: 32767 nh: 268435459
ESI: 03:00:00:00:22:22:22:00:00:01
 Type: Local,Remote
 Interface: ae2
 State: up
 Bridge port: yes
 Ready for BGP: yes
 VNI Count: 2
 MAC Count: 0
 DF status: df 
 DF preference: 32767
 Nexthop group: 536870914
 VTEPs:
     10.226.14.202 df_alg: preference df_pref: 32767 nh: 268435459

Check the VXLAN address table with the following show command.

admin@Leaf3# run show vxlan address-table 
VNID           MAC address          Type       Interface          VTEP
-----------    -----------------    -------    ----------------    ---------------
99             04:f8:f8:20:67:7b    Dynamic                        10.226.14.254  
99             18:5a:58:37:55:e1    Dynamic                        10.226.14.253                         
10020          04:f8:f8:20:67:7b    Dynamic                        10.226.14.253,10.226.14.254    
10030          22:11:11:00:00:00    Dynamic       ae1                                
Entries in access port:  1
Entries in network port: 3

To get the tunnel information within a VNI, run the following show command.

admin@Leaf3# run show vxlan tunnel vni 10030 
Total number of tunnels: 2

VNI 10030, Encap:service-vlan-delete, Decap:service-vlan-per-port
  src addr:10.226.14.202, dst addr:10.226.14.253, state:UP
  traffic type:all
  Vtep type:EVPN 
  nexthops:10.226.15.69 10.226.15.77 
  output ports:ge-1/1/5 ge-1/1/3 

  src addr:10.226.14.202, dst addr:10.226.14.254, state:UP
  traffic type:all
  Vtep type:EVPN 
  nexthops:10.226.15.69 10.226.15.77 
  output ports:ge-1/1/5 ge-1/1/3

To get more info on interfaces belonging to a particular VNI, run the following show command.

admin@Leaf3# run show vxlan vni 10030
Interface      ID             Type           Egress     Vlan ID
-----------    -----------    -----------    -------    --------
ae1            0x80000005    Access          100012     30      
ae2            0x80000006    Access          100013     30      
ge-1/1/23      0x80000007    Access         100014      30      
ge-1/1/5       0x80000008    Network(UC)    200258             
ge-1/1/5       0x8000000a    Network(MC)    100020             
ge-1/1/5       0x8000000b    Network(UC)    200258             
ge-1/1/5       0x8000000d    Network(MC)    100022             
ge-1/1/3       0x8000000f    Network(MC)    100024             
ge-1/1/3       0x80000010    Network(UC)    100017

Show the VXLAN MAC address information on Leaf1.

admin@Leaf1# run show vxlan address-table 
VNID           MAC address          Type       Interface          VTEP
-----------    -----------------    -------    ----------------    ---------------
99             18:5a:58:37:55:e1    Dynamic                        10.226.14.201     
99             70:3e:44:68:22:a4    Dynamic                        10.226.14.202  
10020          04:f8:f8:20:67:7b    Dynamic    ae1 
10030          22:11:11:00:00:00    Dynamic                        10.226.14.201,10.226.14.202                         
Entries in access port:  1
Entries in network port: 3

Show the VXLAN ARP information on Leaf1.

admin@Leaf1# run show vxlan arp
IP-ADDRESS       MAC-ADDRESS        VNI       Status   Age   Interface   REMOTE-VTEP
---------------  -----------------  --------  -------  ----  ----------  ---------------
192.168.0.140    04:f8:f8:20:67:7b  10020     Dynamic  315   ae1
172.168.0.200    22:11:11:00:00:00  10030     Dynamic  177   vplag-0x2   10.226.14.201,10.226.14.202

Show the BGP EVPN route information with command run show bgp evpn route.

EVPN Multihoming with BUM Tunnel

For the topology and figure in the last section, to enable EVPN multihoming with BUM tunnel, besides above configurations of head end replication, we need to add the following configurations:

Configure PIM on each device, and configure one of the devices as static RP (Spine2 in this example).
Configure the same VNI to join the same multicast group.

Leaf1 Configuration

Step 1. Configure PIM and configure Spine2 as static RP.

admin@Leaf1# set protocols pim rp 202.202.202.202 group 224.1.1.1/32
admin@Leaf1# set protocols pim rp 202.202.202.202 group 230.1.1.1/32
admin@Leaf1# set protocols pim interface lo
admin@Leaf1# set protocols pim interface vlan4014
admin@Leaf1# set protocols pim interface vlan4012

Step 2. Configure the same VNI to join the same multicast group.

NOTE:

PICOS supports only one VXLAN segment mapping to one IP multicast group, which is the way to provide the optimal multicast forwarding. That is, to have multiple VXLAN segments share a single IP multicast group in the core network is not supported.

admin@Leaf1# set vxlans vni 10020 mcast-group 224.1.1.1
admin@Leaf1# set vxlans vni 10030 mcast-group 230.1.1.1
admin@Leaf1# commit

Leaf2 Configuration

admin@Leaf2# set protocols pim rp 202.202.202.202 group 224.1.1.1/32
admin@Leaf2# set protocols pim rp 202.202.202.202 group 230.1.1.1/32
admin@Leaf2# set protocols pim interface lo
admin@Leaf2# set protocols pim interface vlan4014
admin@Leaf2# set protocols pim interface vlan4013
admin@Leaf2# set vxlans vni 10020 mcast-group 224.1.1.1
admin@Leaf2# set vxlans vni 10030 mcast-group 230.1.1.1
admin@Leaf2# commit

Leaf3 Configuration

admin@Leaf3# set protocols pim rp 202.202.202.202 group 224.1.1.1/32
admin@Leaf3# set protocols pim rp 202.202.202.202 group 230.1.1.1/32
admin@Leaf3# set protocols pim interface lo
admin@Leaf3# set protocols pim interface vlan4016
admin@Leaf3# set protocols pim interface vlan3012
admin@Leaf3# set vxlans vni 10020 mcast-group 224.1.1.1
admin@Leaf3# set vxlans vni 10030 mcast-group 230.1.1.1
admin@Leaf3# commit

Leaf4 Configuration

admin@Leaf4# set protocols pim rp 202.202.202.202 group 224.1.1.1/32
admin@Leaf4# set protocols pim rp 202.202.202.202 group 230.1.1.1/32
admin@Leaf4# set protocols pim interface lo
admin@Leaf4# set protocols pim interface vlan3011
admin@Leaf4# set protocols pim interface vlan3012
admin@Leaf4# set vxlans vni 10020 mcast-group 224.1.1.1
admin@Leaf4# set vxlans vni 10030 mcast-group 230.1.1.1
admin@Leaf4# commit

Spine1 Configuration

admin@Spine1# set protocols pim rp 202.202.202.202 group 224.1.1.1/32
admin@Spine1# set protocols pim rp 202.202.202.202 group 230.1.1.1/32
admin@Spine1# set protocols pim interface lo
admin@Spine1# set protocols pim interface vlan4011
admin@Spine1# set protocols pim interface vlan4092
admin@Spine1# set protocols pim interface vlan4014
admin@Spine1# set protocols pim interface vlan4013
admin@Spine1# set protocols pim interface vlan4015
admin@Spine1# set protocols pim interface vlan4016
admin@Spine1# commit

Spine2 Configuration

admin@Spine2# set protocols pim rp 202.202.202.202 group 224.1.1.1/32
admin@Spine2# set protocols pim rp 202.202.202.202 group 230.1.1.1/32
admin@Spine2# set protocols pim interface lo
admin@Spine2# set protocols pim interface vlan4011
admin@Spine2# set protocols pim interface vlan4092
admin@Spine2# set protocols pim interface vlan4012
admin@Spine2# set protocols pim interface vlan4014
admin@Spine2# set protocols pim interface vlan4015
admin@Spine2# commit

Verify Configuration

In addition to the above show command to verify the results, you can also use the run show vxlan mcast-tunnel vni command to view the establishment of the BUM tunnel.

The following example shows the result on Leaf 3:

admin@Leaf3# run show vxlan mcast-tunnel vni 10020
VNI       Src-addr         Mcast-addr       Out-Vlan  Out-Interface    Tunnel-ID     Egress-ID
--------  ---------------  ---------------  --------  ---------------  ------------  ------------
10020     10.226.14.201    224.1.1.1        3012      ge-1/1/5         0x4c000000    0           

admin@Leaf3# run show vxlan mcast-tunnel vni 10030
VNI       Src-addr         Mcast-addr       Out-Vlan  Out-Interface    Tunnel-ID     Egress-ID
--------  ---------------  ---------------  --------  ---------------  ------------  ------------
10030     10.226.14.201    230.1.1.1        3012      ge-1/1/5         0x4c000001    0

PICOS-4.4.5_Configuration_Guide

EVPN Multihoming Configuration Guide

Introduction

Key Benefits

Brief Mechanism

EVPN Type-1 Ethernet Auto Discovery (A-D) Route

EVPN Type-4 Ethernet Segment Route

Designated Forwarder Election

VTEP Uplink Status Tracking

EVPN MH Split Horizon

EVPN MH Basic Configuration

Aliasing

Switch Configuration Example and Topology

EVPN Multihoming with Head End Replication

Leaf1 Configuration

Leaf2 Configuration

Leaf3 Configuration

Leaf4 Configuration

Spine1 Configuration

Spine2 Configuration

Verify Configuration

EVPN Multihoming with BUM Tunnel

Leaf1 Configuration

Leaf2 Configuration

Leaf3 Configuration

Leaf4 Configuration

Spine1 Configuration

Spine2 Configuration

Verify Configuration