Introduction

Ethernet Virtual Private Network or EVPN is a technology designed to carry Layer 2 traffic over wide area network protocols. EVPN is a multi-tenant BGP-based control plane for layer-2 (bridging) and layer-3 (routing) VPNs. It’s the unifying L2+L3 equivalent of the traditional L3-only MPLS/VPN control plane. PICOS EVPN implementation leverages VXLAN technology as described in RFC7348.

VXLAN has been the predominant technology used in the enterprise and data center domains to achieve Layer 2 level scalability over an IP overlay backbone. VXLAN has become the technology of choice for separating the virtual network from underlying physical network and has greatly enhanced the network virtualization, easier network management and orchestration. VXLANs provides network segmentation but also helps solve the scalability issue normally associated with VLANs.

The following list describes the list of features that PiCOS BGP EVPN implementation supports.

1. Exchange of VNI membership between VTEPs using EVPN type 3 routes.
2. Exchange of host MAC and IP addresses using EVPN type 2 routes.
3. Exchange of MAC Mobility Extended Community to support host/VM mobility.
4. Dual attached host via VXLAN active-active mode. MAC synchronization between switches is achieved via MLAG.
5. Inter Subnet routing for IPv4. Distributed symmetric and asymmetric routing between different subnets and centralized routing.
6. Prefix-based routing using EVPN type-5 routes (EVPN IP prefix route).
7. Multi-tenancy over layer 3.

Both eBGP and iBGP peerings can be used for the EVPN address family. 

EVPN Fundamental Configuration

The following steps represent the fundamental configuration to use EVPN as the control plane for VXLAN. These steps are in addition to configuring VXLAN interfaces, attaching them to a bridge, and mapping VLANs to VNIs.

1. Configure Physical interfaces and assign VLANs to interfaces.
2. Configure L3 interfaces and assign IP addresses to interfaces.
3. Configure VXLAN VNIs and enable VXLAN VNI mapping to VLAN IDs.
4. Enable EVPN route exchange (that is, address-family layer 2 VPN/EVPN) between BGP peers.
5. Enable EVPN on the system to advertise VNIs and host reachability information (MAC addresses learned on associated VLANs) to BGP peers.

Additional configuration is necessary such as provision of inter-subnet routing. The configuration depends on the deployment scenario. You can also configure various other BGP parameters depending on your network requirements.

Enable EVPN Between BGP Peers

The very basic steps needed to enable BGP EVPN between a BGP neighbor is as under.

admin@router1# set protocols bgp local-as 65101
admin@router1# set protocols bgp router-id 10.10.10.1
admin@router1# set protocols bgp neighbor 100.1.1.134 remote-as 650101
admin@router1# commit
Commit OK.
Save Done!

The configuration below adds the evpn address family to the BGP neighbor address-family so that BGP peers activate exchanging EVPN routes with each other. After this configuration, the BGP still does not know about the local VNI's. 

Advertise All VNIs Through BGP

To allow BGP know about all VNIs or hosts associated with those local VNIs, enable the BGP control plane for all VNIs using the configuration shown below.

admin@router# set protocol bgp evpn advertise-all-vni
admin@router# commit
Commit OK.
Save Done!

EVPN MAC Learning Process

In Figure 1, when Host 1 is first plugged into R1, Host 1 will start sending ARP and other basic networking traffic like DHCP. When R1 receives a packet from Host 1 for the first time, it will record its MAC address in its local MAC address table. Also, R1 will advertise an EVPN Type-2 route to R2. The route includes the local EVPN instance of R1, the VTEP IP address, the Host 1 MAC address and the L2VNI.

Upon receiving the EVPN Type-2 route from R1, R2 learns the MAC address of Host 1. To accept this route, R2 needs to determine if the Import Route Target (IRT) configured on R2 matches the Export Route Target (ERT). RT is sent as the BGP Extended Community attribute. In this case the IRT and ERT match hence the route is accepted and the MAC address of Host 1 is learned.

                                               Figure 1.  MAC Learning and Packet Forwarding

Packet Forwarding Process

In the case of packet forwarding within the same subnet as depicted in Figure 2, both Host1 and Host2 belong to the same VNI. Host1 wants to send a packet to Host2.

1. If Host1 doesn't have the MAC address of Host1, the MAC address can be learned through the MAC learning process described in the section above. Assuming that Host1 does have the MAC address of Host2, Host 1 sends the packet to R1 destined for Host2.
2. R1 receives the packet and determines the VNI of Host1 based on the ingress interface configuration. R1 learned the Host2 MAC address and the outgoing interface is the VTEP 2.2.2.2 on R2.
3. R1 then encapsulates the original packet from Host1 with the VXLAN header and sends it out.
4. When the packet is received on R2, the outer VXLAN header is stripped off. R2 then searches its local MAC table and finds out the out going interface and delivers the original packet to Host2.

ARP/ND Suppression

set vlans vlan-id 10
set vxlans vni 10010 vlan 10
set vlans vlan-id 10 l3-interface vlan10
set l3-interface vlan-interface vlan10
set vxlans vni 10010 arp-nd-suppress disable false

In MP-BGP EVPN networks, in order to suppress network storms caused by ARP/ND broadcast message flooding, ARP/ND suppression function can be enabled on VTEP devices to reduce network traffic.

In BGP EVPN networks, VTEPs have the ability to learn both local and remote hosts. When a VTEP learns about a local host from Gratuitous ARP or Reverse ARP, the VTEP will locally record the hosts’ MAC and IP address in an ARP Cache Table for that particular VNI. This host MAC and IP address will be shared with remote VTEPs within the same VNI using MP-BGP EVPN Type-2 routes.

When an ARP request broadcast message is received from a host, the local VTEP device with ARP suppression enabled, will actively intercept the message, search the corresponding destination MAC in the VXLAN ARP Cache table, and reply to the ARP message on behalf of the destination host to prevent flooding the entire VXLAN network VNI.

ARP and ND suppression use the same mechanism hence ND suppression is not discussed here.

By default, the ARP/ND suppression function is disabled. Users can use the following command to enable ARP/ND suppression function on VTEP devices.

set vxlans vni <vni-id> arp-nd-suppress disable <true | false>

ARP/ND suppression can be enabled on L2 VNI and works only on ARP/ND broadcast messages in the corresponding VNI enabled with this function.

Remove EVPN Configuration

Warning:

To correctly remove EVPN configuration, the following deletion sequence needs to be followed.

1. First, remove the configuration of the data plane, that is, VXLAN-related configuration, including commands under the following configuration node: set vxlans xxx.
2. Then, remove the control plane configuration, i.e., BGP and BGP EVPN related configurations, including commands under the following configuration node: set protocols bgp xxx.
3. Last, delete VXLAN related L3 interface configuration, including the following commands:

          set l3-interface vlan-interface xxx

         set l3-interface routed-interface xxx

If you don’t follow this deletion order, the configuration in the kernel may still exist after the deletion operation, inconsistency in two systems (PICOS and kernel) may lead to unpredictable problems.