Configuring VXLAN

Only switches with ASIC Trident-II (e.g. P5401 and P5101) can support VXLAN, and the port number of VXLAN ranges from 4097 to 5119. VXLAN mechanism is based on the limited number of VLANs(0-4094).VXLAN is used to provide more networks for switches or the host. To resolve the problem that pushing the interface' PVID to the untagged packets before encapsulated by the VXLAN header, the user must use this command "ovs-vsctl set interface <interface> type=pica8 options:access-vport=true ". Like this, the untagged packets can be encapsulated by VXLAN header with no VLAN that is pvid of ingress port. And the tagged packets are encapsulated by VXLAN header, the inner VLAN is the VLAN tag of packets that received by ingress port.

1, The port ranges in PicOS are as follows:

Port Type	Port Number
VXLAN	4097-5119

2.When the vxlan packets wants to be decapsulated,the mac address(src/dst) in the flow's match means the inner packets' mac address.Other match fields like dl_vlan is the vxlan packets' vlan.

eg:ovs-ofctl add-flow br0 in_port=4097,dl_vlan=1,dl_src=22:11:11:11:11:11,dl_dst=22:22:22:22:22:22,actions=2

3.The vxlan encapsulation flow do not support multiple ingress port(like lag,bond).Because vxlan encapsulation is in vcap,and vcap can not suport multiple ports.

4.Vxlan decapsulation flow's action also support group(type=all,lag-select-group),do not support modify mac or vlan before vxlan port 4097.

Command

admin@PicOS-OVS$ovs-vsctl add-port br0 vxlan1 -- set interface vxlan1 type=pica8_vxlan options:remote_ip=10.10.10.2 options:local_ip=10.10.10.1 options:vlan=1 options:vnid=1122867 options:udp_dst_port=4789 options:src_mac=C8:0A:A9:04:49:1A  options:dst_mac=C8:0A:A9:9E:14:A5  options:egress_port=qe-1/1/2

Description

br0: bridge name
remote_ip=10.10.10.2: the IP address of the peer VXLAN tunnel interface; this IP address will be the destination IP of the encapsulated VXLAN packets
local_ip=10.10.10.1: the IP address of this VXLAN tunnel interface; this IP address will be the source IP of the encapsulated VXLAN packets
src_mac==C8:0A:A9:9E:49:1A: the logical MAC address of the VXLAN tunnel interface; this MAC address will be the source MAC of the encapsulated VXLAN packets to next-hop
dst_mac=C8:0A:A9:9E:14:A5: the next-hop MAC address; this MAC address will be the destination MAC the encapsulated VXLAN packets to next-hop
egress_port=qe-1/1/2: the output port of the encapsulated VXLAN packets
vnid=1234:the key value of VXLAN tunnel,different tunnel has different vnid.
vlan=1:the vlan of VXLAN tunnel.this vlan will be pop or not according to the pvid of the egress port.
udp_dst_port=4789:the udp destination port of encapsulating packets by VXLAN tunnel.All the encapsulated packets has the this udp dst port.

Examples

configure a VXLAN tunnel

topology

configuration

(1)create a new bridge named br0.

admin@PicOS-OVS$ovs-vsctl add-br br0 -- set bridge br0 datapath_type=pica8

(2)add ports to br0.

admin@PicOS-OVS$ovs-vsctl add-port br0 qe-1/1/1 vlan_mode=trunk tag=1  -- set Interface qe-1/1/1 type=pica8
admin@PicOS-OVS$ovs-vsctl add-port br0 qe-1/1/2 vlan_mode=trunk tag=1  -- set Interface qe-1/1/2 type=pica8

(3)add a VXLAN port named vxlan1 on qe-1/1/2

admin@PicOS-OVS$ovs-vsctl add-port br0 vxlan1 -- set interface vxlan1 type=pica8_vxlan options:remote_ip=10.10.10.2 options:local_ip=10.10.10.1 options:vlan=1 options:vnid=1122867 options:udp_dst_port=4789 options:src_mac=C8:0A:A9:04:49:1A  options:dst_mac=C8:0A:A9:9E:14:A5  options:egress_port=qe-1/1/2

add a flow to switch

admin@PicOS-OVS$ovs-ofctl  add-flow br0 in_port=1,actions=output:4097

Send packets to qe-1/1/1,qe-1/1/2 will receive pakcets that encapsulated by VXLAN header. When VLAN of VXLAN tunnel is the same with the pvid of qe-1/1/2,the packets from qe-1/1/2 will be stripped VLAN of VXLAN. Or, packets will have two VLANs (outer VLAN is vxlan-vlan, inner VLAN is the pvid of ingress port or original VLAN of packets)

strip a VXLAN header

topology

configuration

User must configure VXLAN port on qe-1/1/2 and qe-1/1/3,and add some flows to the switches so that packets can be encapsulated or decapsulated and forwarded correctly.

(1) create a new bridge named br0.

admin@PicOS-OVS$ovs-vsctl add-br br0 -- set bridge br0 datapath_type=pica8

(2) add ports to br0.

SwA:

admin@PicOS-OVS$ovs-vsctl add-port br0 qe-1/1/1 vlan_mode=trunk tag=1  -- set Interface qe-1/1/1 type=pica8
admin@PicOS-OVS$ovs-vsctl add-port br0 qe-1/1/2 vlan_mode=trunk tag=1  -- set Interface qe-1/1/2 type=pica8

SWb:

admin@PicOS-OVS$ovs-vsctl add-port br0 qe-1/1/1 vlan_mode=trunk tag=1  -- set Interface qe-1/1/1 type=pica8
admin@PicOS-OVS$ovs-vsctl add-port br0 qe-1/1/2 vlan_mode=trunk tag=1  -- set Interface qe-1/1/2 type=pica8

(3) add VXLAN port vxlan1 on egress port qe-1/1/2 of switcha and switchb

Swa:

admin@PicOS-OVS$ovs-vsctl add-port br0 vxlan1 -- set interface vxlan1 type=pica8_vxlan options:remote_ip=10.10.10.2 options:local_ip=10.10.10.1 options:vlan=1 options:vnid=1122867 options:udp_dst_port=4789  options:src_mac=C8:0A:A9:04:49:1A  options:dst_mac=C8:0A:A9:9E:14:A5  options:egress_port=qe-1/1/2

flow in swa.

admin@PicOS-OVS$ovs-ofctl  add-flow br0 in_port=1,actions=output:4097

Swb:

admin@PicOS-OVS$ovs-vsctl add-port br0 vxlan1 -- set interface vxlan1 type=pica8_vxlan options:remote_ip=10.10.10.1 options:local_ip=10.10.10.2 options:vlan=1 options:vnid=1122867 options:udp_dst_port=4789  options:src_mac= C8:0A:A9:9E:14:A5  options:dst_mac= C8:0A:A9:04:49:1A  options:egress_port=qe-1/1/2

admin@PicOS-OVS$ovs-ofctl add-flow br0 in_port=4097,actions= output:1

send packets to qe-1/1/1 of swa,qe-1/1/1 of switchb will receive the orignal packets( the contents of packets are the same with packets that qe-1/1/1 of swa received).

configure two VXLAN tunnels on a pair of physical port

topology

configuration

add two pairs of VXLAN ports on qe-1/1/2,qe-1/1/3

sw1:

admin@PicOS-OVS$ovs-vsctl add-port br0 vxlan1 -- set interface vxlan1 type=pica8_vxlan options:remote_ip=10.10.10.2 options:local_ip=10.10.10.1 options:vlan=1 options:vnid=1122867 options:udp_dst_port=4789  options:src_mac=C8:0A:A9:04:49:1A  options:dst_mac=C8:0A:A9:9E:14:A5  options:egress_port=qe-1/1/2
admin@PicOS-OVS$
admin@PicOS-OVS$ovs-vsctl add-port br0 vxlan2 -- set interface vxlan2 type=pica8_vxlan options:remote_ip=10.10.60.1 options:local_ip=10.10.60.2 options:vlan=2 options:vnid=1122869 options:udp_dst_port=4789  options:src_mac=22:22:22:04:49:1A  options:dst_mac=44:44:44:9E:14:A5  options:egress_port=qe-1/1/2

flows in sw1,

admin@PicOS-OVS$ovs-ofctl add-flow br0 in_port=1,dl_src=22:22:22:22:22:22,actions=output:4097  
admin@PicOS-OVS$ovs-ofctl add-flow br0 in_port=1,dl_src=22:22:22:22:22:23,actions=output:4098

sw2:

admin@PicOS-OVS$ovs-vsctl add-port br0 vxlan1 -- set interface vxlan1 type=pica8_vxlan options:remote_ip=10.10.10.1 options:local_ip=10.10.10.2 options:vlan=1 options:vnid=1122867 options:udp_dst_port=4789 options:src_mac=C8:0A:A9:9E:14:A5  options:dst_mac=C8:0A:A9:04:49:1A options:egress_port=qe-1/1/3
admin@PicOS-OVS$ovs-vsctl add-port br0 vxlan2 -- set interface vxlan2 type=pica8_vxlan options:remote_ip=10.10.60.2 options:local_ip=10.10.60.1 options:vlan=2 options:vnid=1122869 options:udp_dst_port=4789  options:src_mac=44:44:44:04:49:1A  options:dst_mac=22:22:22:9E:14:A5  options:egress_port=qe-1/1/3

flows in sw2,

admin@PicOS-OVS$ovs-ofctl add-flow br0 in_port=4097,dl_src=22:22:22:22:22:22,actions=output:4
admin@PicOS-OVS$ovs-ofctl add-flow br0 in_port=4098,dl_src=22:22:22:22:22:23,actions=output:5

send packets to qe-1/1/1 of sw1,qe-1/1/4 should receive the packets with src_mac :22:22:22:22:22:22,and qe-1/1/5 should receive the packets with src_mac 22:22:22:22:22:23.

collaboration between L2GRE and VXLAN

topology

configuration

User must configure VXLAN port and l2gre port on qe-1/1/2 and qe-1/1/3.Add flows on both switches, so packets can be forwarded correctly.

sw1:

admin@PicOS-OVS$ovs-vsctl add-port br0 vxlan1 -- set interface vxlan1 type=pica8_vxlan options:remote_ip=10.10.10.2 options:local_ip=10.10.10.1 options:vlan=1 options:vnid=1122867 options:udp_dst_port=4789  options:src_mac=C8:0A:A9:04:49:1A  options:dst_mac=C8:0A:A9:9E:14:A5  options:egress_port=qe-1/1/2
admin@PicOS-OVS$
ovs-vsctl add-port br0 l2gre1 -- set Interface l2gre1 type=pica8_l2gre options:remote_ip=10.10.61.10 options:local_ip=10.10.60.10 options:vlan=1 options:l2gre_key=1234 options:src_mac=C8:0A:A9:22:22:22 options:dst_mac=C8:0A:A9:33:33:33  options:egress_port=qe-1/1/2

flows in sw1,

admin@PicOS-OVS$ovs-ofctl add-flow br0 in_port=1,dl_dst=22:22:22:22:22:22,actions=output:4097
admin@PicOS-OVS$ovs-ofctl add-flow br0 in_port=1,dl_dst=22:22:22:22:22:23,actions=output:5121

sw2:

admin@PicOS-OVS$ovs-vsctl add-port br0 vxlan1 -- set interface vxlan1 type=pica8_vxlan options:remote_ip=10.10.10.1 options:local_ip=10.10.10.2 options:vlan=1 options:vnid=1122867 options:udp_dst_port=4789  options:src_mac=C8:0A:A9:9E:14:A5 options:dst_mac=C8:0A:A9:04:49:1A options:egress_port=qe-1/1/3
admin@PicOS-OVS$ovs-vsctl add-port br0 l2gre1 -- set Interface l2gre1 type=pica8_l2gre options:remote_ip=10.10.60.10 options:local_ip=10.10.61.10 options:vlan=1 options:l2gre_key=1234 options:src_mac=C8:0A:A9:33:33:33 options:dst_mac=C8:0A:A9:22:22:22 options:egress_port=qe-1/1/3

flows in sw2,

admin@PicOS-OVS$ovs-ofctl  add-flow br0 in_port=4097,dl_dst=22:22:22:22:22:22,actions=output:4
admin@PicOS-OVS$ovs-ofctl  add-flow br0 in_port=5121,dl_dst=22:22:22:22:22:23,actions=output:5

Vnid must be the same when the user wants to build a VXLAN tunnel between two ports. Different VXLAN tunnels must have different vnids. Besides, packets are not decapsulated when the vnid is different between the VXLAN tunnel. VXLAN can work together with GRE, L2GRE, VXLAN.

Option:

topology

Generally, untagged packets from Host A sent through Swa will be tagged by pvid in port qe-1/1/1. The new tagged packet adds VXLAN header and strip VXLAN header through VXLAN tunnel, and will keep the tag while forwarding on Swb qe-1/1/1 even though the tag equals the pvid of Swb qe-1/1/1. The result is that Host B receives a tagged packet which is different from the original packet.

To avoid the above issue, pica8 support packet keep untag through pica8 switch port. The following command is necessary.

ovs-vsctl set interface qe-1/1/1 options:access-vport=true

If the user adds the command on Swa, untagged packets that pass through Swa qe-1/1/1 will not be tagged by the pvid, then pass through VXLAN tunnel and stay untagged while forwarding to Swb qe-1/1/1. The result is Host B will receive untag packet.

vxlan numbers

User can create 1023 VXLAN ports at most (with the same mac, ip, vlan, only vnid is different) on one physical port. But the max flows number of VXLAN flow (with action=output:4097) is the minimum value of flows number that Vfilter table and Tcam table can support. That is 512 on P5401, P5101, AS6701, P5712,2632,tomahawk chip

Configure VxLAN Ecmp

From version 2.9.1, Pica8 switch support configuring Vxlan ecmp.That is to say,the traffic can hash to one or more ports of the ecmp.So the l3-ecmp-hash-fields is also valid to vxlan ecmp.And the egress port number suported is controlled by "ovs-vsctl set-max-ecmp-ports ",could be 2,4,8,16,32,default value is 4.

Command

ovs-vsctl add-port br0 vxlan1 -- set interface vxlan1 type=pica8_vxlan options:remote_ip=10.10.10.2 options:local_ip=10.10.10.1 options:vnid=1122867 options:udp_dst_port=4789 options:vlan=1 options:src_mac=C8:0A:A9:04:49:1A options:dst_mac=C8:0A:A9:9E:14:A5 options:egress_port=te-1/1/2 options:vlan2=2 options:src_mac2=C8:0A:A9:04:49:1B options:dst_mac2=C8:0A:A9:9E:14:A6 options:egress_port2=te-1/1/3 options:vlan3=3 options:src_mac3=C8:0A:A9:04:49:1C options:dst_mac3=C8:0A:A9:9E:14:A7 options:egress_port3=te-1/1/4

the usage is the same with vxlan.

.