/
VXLAN Configuration Guide

VXLAN Configuration Guide

Owned by Tracy Yang (Unlicensed)
Last updated: Mar 19, 2021

NOTES:

  • The switch platforms which use this feature are:
    • Trident2
    • Trident2+
    • Tomahawk
    • Tomahawk+
    • Tomahawk 2
    • Tomahawk 3
    • Trident3
    • Maverick
  • If VXLAN is deployed in an MLAG domain, it behaves a little differently. For details, see MLAG Configuration.

  • Open vSwitch Database (OVSDB) management protocol runs in management VRF by default. If you use Ethernet management interface Eth0/1 to connect with the controller, you do not need to do the following operations.

    However, if the L3 VLAN interface is used to connect with the controller, the OVSDB management protocol cannot run normally by default, as all the L3 VLAN interface is in the default VRF by default. You need to modify the configurations to make the L3 VLAN interface management interface and OVSDB management protocol run in the same VRF, so as to run the OVSDB management protocol normally. You can choose either one of the following two ways:

    Way 1. Use the command set protocols ovsdb controller <controller-name> vrf default to move OVSDB management protocol to run in the default VRF. This way is recommended.

    Way 2. Use the command set vlan-interface interface <interface-name> vrf mgmt-vrf to move the L3 VLAN interface connected to the controller to management VRF.

  • VXLAN is allowed to be configured on the ports with NAC.

About VXLAN

Virtual Extensible LAN (VXLAN) is an overlay network virtualization technology. An overlay network is a virtual network that is built on top of existing network Layer 2 and Layer 3 technologies to support elastic compute architectures. VXLAN makes it easier for network engineers to scale out a cloud computing environment while logically isolating cloud apps and tenants.

VXLAN Technology

VXLAN uses UDP-based encapsulation to tunnel Ethernet frames and transfers original data packets as tunnel payloads. With the outer UDP tunnel, inner payload data can be quickly transferred on the layer 2 and layer 3 networks. To provide the capability of broadcast domain addressing, the VXLAN technology uses layer 3 IP multicast to replace the Ethernet broadcast. Therefore, the broadcast, unknown unicast, and multicast (BUM) packets can be transferred on virtual networks through broadcasting. For more VXLAN details, please read RFC7348.

VXLAN Standards

As shown in Figure 1-1, a VXLAN packet consists of the outer encapsulation and the inner payloads.

  • Flags (8 bits): The flag I must be set to 1 for a valid VXLAN Network Identifier (VNI). The other 7 bits (labeled as R) are reserved fields and must be set to 0 on transmit and ignored on receive.
  • VXLAN segment ID or VXLAN VNI: This parameter contains 24 bits and is used to designate the individual VXLAN overlay network on which the VMs are located.
  • Reserved fields (24 bits and 8 bits): This parameter must be set to 0 on transmit and ignored on receive.
  • The destination port number assigned to the outer tunnel is 4789, which is dedicated.  

VXLAN Inner 802.1Q

Encapsulation mode

Encapsulation means the flow from access ports to network ports. Use one of the following options to specify actions about 802.1Q tag while encapsulation.

  • none: Nothing will change, untagged packets will stay untagged, tagged packets will stay tagged.
  • service-vlan-add: Add 802.1Q tag for untagged packets, and nothing changed with tagged packets. Encapsulation vlan is required.
  • service-vlan-add-delete: Add 802.1Q tag for untagged packets, and delete tag for tagged packets. Encapsulation vlan is required.
  • service-vlan-add-replace: Add 802.1Q tag for untagged packets, and replace tag for tagged packets. Encapsulation vlan is required.
  • service-vlan-delete: Delete 802.1Q tag for tagged packets, and nothing changed with untagged packets. This is default value according to RFC 7348.
  • service-vlan-replace: Replace vlan id of 802.1Q tag for tagged packets, and nothing changed with untagged packets. Encapsulation vlan is required.

Decapsulation-mode

Decapsulation means the flow from network ports to access ports. Decapsulation-mode configuration takes effect in non OVSDB VTEP scenarios.

  • none: Nothing will change, untagged packets will stay untagged, tagged packets will stay tagged.
  • service-vlan-add: From network ports to access ports, add 802.1Q tag for both untagged/tagged packets. If the access port is matched by port and vlan, the vlan id of the tag being added will be that vlan, otherwise will be PVID of that port.
  • service-vlan-add-delete: From network ports to access ports, add 802.1Q tag for both untagged/tagged packets. If the access port is matched by port and vlan, the vlan id of the tag being added will be that vlan, otherwise will be PVID of that port. From access to access, delete tag for tagged packets.
  • service-vlan-add-replace: From network ports to access ports, add 802.1Q tag for both untagged/tagged packets. If the access port is matched by port and vlan, the vlan id of the tag being added will be that vlan, otherwise will be PVID of that port. From access to access, replace tag for tagged packets.This is the default value.
  • service-vlan-delete: From access to access, delete tag for tagged packets.
  • service-vlan-replace: From access to access, replace tag for tagged packets.

Based on the above description, please see the following three tables for the detailed traffic changes.

The below table shows the traffic changes in the case that interfaces in the access side are binded with a vxlan in the network side.


Access->Access

Access->Network

Network->Access

none

untag-->untag
tag-->remain tag

untag->untag

tag->remain tag

untag-->untag
tag-->remain tag

service-vlan-add

untag->tag(add pvid)

tag->remain tag

untag->add encapsulation vlan tag

tag->remain tag

 untag-->tag(add pivd)

tag-->double tag(outer layer add pvid)

service-vlan-add-delete

 untag->tag(add pvid)

tag-->untag

 untag-->add encapsulation vlan

tag->untag(been deleted)

 untag-->tag(add pivd)

tag-->double tag(outer layer add pvid)

service-vlan-add-replace

 untag->tag(add pvid)

tag-->new tag(replaced with pvid)

 untag-->add encapsulation vlan

tag->tag(changed to encapsulation vlan)

 untag-->tag(add pivd)

tag-->double tag(outer layer add pvid)

service-vlan-delete

 untag-->untag

tag-->untag

 untag->untag

tag->untag

 untag-->untag

tag-->remain tag

service-vlan-replace

 untag-->untag

tag-->new tag(replaced with pvid)

 untag->untag

tag->tag(changed to encapsulation vlan)

 untag-->untag

tag-->remain tag


The below table shows the traffic changes in the case that the interfaces and vlans in the access side are binded with a vxlan in the network side.


Access->Access

Access->Network

Network->Access

none

tag-->remain tag

tag->remain tag

untag-->untag

tag-->remain tag

service-vlan-add

tag->remain tag

tag->remain tag

untag-->tag(add vxlan-vlan)

tag-->double tag(outer layer add vxlan-vlan)

service-vlan-add-delete

tag-->untag

tag->untag(been deleted)

untag-->tag(add vxlan-vlan)

tag-->double tag(outer layer add vxlan-vlan)

service-vlan-add-replace

tag-->new tag(replaced with vxlan-vlan)

tag->tag(changed to encapsulation vlan)

untag-->tag(add vxlan-vlan)

tag-->double tag(outer layer add vxlan-vlan)

service-vlan-delete

tag-->untag

tag->untag

untag-->untag

tag-->remain tag

service-vlan-replace

tag-->new tag(replaced with vxlan-vlan)

tag->tag(changed to encapsulation vlan)

untag-->untag

tag-->remain tag


The below table shows the traffic changes in the case that only vlans in the access side are binded with a vxlan in the network side.


Access->Access

Access->Network

Network->Access

none

tag-->remain tag

tag->remain tag

untag-->untag

tag-->remain tag

service-vlan-add

tag->remain tag

tag->remain tag

untag-->tag(add vxlan-vlan)

tag-->double tag(outer layer add vxlan-vlan)

service-vlan-add-delete

tag-->untag

tag->untag(been deleted)

untag-->tag(add vxlan-vlan)

tag-->double tag(outer layer add vxlan-vlan)

service-vlan-add-replace

tag-->remain tag

tag->tag(changed to encapsulation vlan)

untag-->tag(add vxlan-vlan)

tag-->double tag(outer layer add vxlan-vlan)

service-vlan-delete

tag-->untag

tag->untag

untag-->untag

tag-->remain tag

service-vlan-replace

tag-->remain tag

tag->tag(changed to encapsulation vlan)

untag-->untag

tag-->remain tag

VXLAN ECMP

In L2/L3, VXLAN ECMP is supported. Picos supports up to 32-way ECMP.

  • The VXLAN  ECMP does not need special configuration. It entirely depends on the routing ECMP. The route ECMP configure link: ECMP (Equal-Cost Multipath Routing) Configuration
  • PicOS uses info from VXLAN header for hash calculation to ensure better performance.

VXLAN Mac Learning

The VTEP performs source MAC learning on the VNI as a Layer 2 switch.

  • The switch receives traffic from the local VTEP to the remote VTEP, the VTEP learns the source MAC address in the access port.
  • The switch receives traffic from the remote VTEP to the local VTEP, the VTEP learns the source MAC address in the network port.

A VNI MAC address table includes the following types of MAC address entries:

  • Access port--Dynamic MAC address entries learned from the local VTEP. VXLAN does not support local configure static MAC address.
  • Network port--Include static and dynamic MAC entries.

Static mac--Configure static mac address entries on VXLAN tunnel interfaces.
Dynamic mac--The MAC address entries learned from incoming traffic on VXLAN tunnels. The learned MAC addresses are contained in the inner Ethernet header source MAC.

On network port, the configure static mac entry has higher priority than dynamic mac entries.

VXLAN Traffic Forwarding

Unicast Traffic

  • The switch receives traffic from the access port. The VTEP encapsulates the original Ethernet frame with an outer MAC header, outer IP header, and a VXLAN header. The source IP address is the source VTEP's VXLAN tunnel source IP address.
  • The local VTEP forwards the encapsulates packets to the VXLAN tunnel a destination IP address.
  • The remote VTEP decapsulates the packet and forwards the frame to access port.

Broadcast and Unknown Traffic

  • The switch receives traffic from the access port. The VTEP encapsulates the original Ethernet frame with an outer MAC header, outer IP header and a VXLAN header. The source IP address is the source VTEP's VXLAN tunnel source IP address.
  • The local VTEP flood encapsulates packets to the VXLAN tunnel all destination IP address.
  • The all remote VTEP decapsulates the packet and forwards the frame to access port.

Configure map port to VXLAN VNI or map port&vlan to VXLAN VNI Step

VxLAN supported on PicOS L2/L3 switch, the commands configure step as below.

  • Configure VXLAN soure interface

    set vxlans source-interface loopback address 10.10.10.25
commit

  • Create VXLAN VNI

    set vxlans vni 10010
commit

  • Configure vtep address for VXLAN VNI

    set vxlans vni 10010 flood vtep 10.10.10.12
commit

  • Add VXLAN port into VXLAN VNI

    set vxlans vni 10010 interface te-1/1/40 vlan 100
commit

Configure map vlan to VXLAN VNI Step

VxLAN supported on PicOS L2/L3 switch. To configure Step, pleae see below.

  • Configure VXLAN soure interface

    set vxlans source-interface loopback address 10.10.10.25
commit


  • Create VXLAN VNI

    set vxlans vni 10010
commit

  • Configure vtep address for VXLAN VNI

    set vxlans vni 10010 flood vtep 10.10.10.12
commit

  • Enable VXLAN VNI map with vlan

    set vxlans vni-map-vlan true
commit

  • Add vlan into VXLAN VNI

    set vxlans vni 10010 vlan 100
commit


Copyright © 2025 Pica8 Inc. All Rights Reserved.