PICOS OVS Troubleshooting
- Tracy Yang
Owned by Tracy Yang
May 24, 2024
6 min read
Loading data...
This section details basic procedures to troubleshoot PicOS switches in OVS (Open vSwitch) mode.
Verifying PicOS Mode
Verify if PicOS is actually running in OVS (Open vSwitch) mode, as described in Checking PicOS Mode.
When PicOS is running in the OVS mode, two processes should be running: ovsdb-server and ovs-vswitchd.
admin@XorPlus$ps -ef | grep ovs root 1356 1 0 Jan26 ? 00:00:10 /ovs/sbin/ovsdb-server /ovs/ovs-vswitchd.conf.db --pidfile --remote=punix:/ovs/var/run/openvswitch/db.sock root 1358 1 0 Jan26 ? 00:19:07 /ovs/sbin/ovs-vswitchd --enable-shared-lcmgr
In CrossFlow mode, the router stack must have been initialized in addition to having ovsdb-server and ovs-vswitchd processes running.
admin@XorPlus$ps -ef | grep pica root 12430 1 0 Jan07 ? 00:05:49 pica_cardmgr root 12432 1 0 Jan07 ? 01:03:19 pica_sif root 12439 1 0 Jan07 ? 00:08:45 pica_lacp root 12441 1 19 Jan07 ? 4-10:50:14 pica_lcmgr root 12447 1 0 Jan07 ? 00:09:58 pica_login root 13218 1 0 Jan07 ? 00:20:47 pica_mstp root 13236 1 0 Jan07 ? 01:25:30 /pica/bin/xorp_rtrmgr -d -L local0.info -P /var/run/xorp_rtrmgr.pid
Verifying Bridge Configuration
For the bridge and ports to forward frames in hardware, the datapath_type configured for each entity must be set to pica8.
admin@PicOS-OVS$ovs-vsctl show
ac9e5b1e-4234-4158-9214-5660b9343779
Bridge east
Controller "tcp:172.16.0.142:6653"
is_connected: true
fail_mode: standalone
Port "ae1"
tag: 1
Interface "ae1"
type: "pica8_lag"
options: {lacp-mode=active, lacp-system-priority="32768", lacp-time=slow, lag_type=lacp, link_speed=auto, members="te-1/1/2"}
Port "te-1/1/2"
tag: 1
Interface "te-1/1/2"
type: "pica8"
options: {flow_ctl=none, link_speed=auto}
Port "te-1/1/1"
tag: 1
Interface "te-1/1/1"
type: "pica8"
options: {flow_ctl=none, link_speed=auto}
admin@PicOS-OVS$ovs-ofctl show east
OFPT_FEATURES_REPLY (OF1.4) (xid=0x2): dpid:1deb0ae61be44040
n_tables:254, n_buffers:256
capabilities: FLOW_STATS TABLE_STATS PORT_STATS GROUP_STATS
OFPST_PORT_DESC reply (OF1.4) (xid=0x4):
1(te-1/1/1): addr:ff:ff:ff:ff:ff:00
config: 0
state: LINK_UP
current: 1GB-FD COPPER
advertised: 1GB-FD 10GB-FD FIBER
supported: 10MB-FD 100MB-FD 1GB-FD 10GB-FD FIBER AUTO_NEG
speed: 1000 Mbps now, 10000 Mbps max
2(te-1/1/2): addr:ff:ff:ff:ff:ff:00
config: 0
state: LINK_DOWN
current: 1GB-FD COPPER
advertised: 1GB-FD 10GB-FD FIBER
supported: 10MB-FD 100MB-FD 1GB-FD 10GB-FD FIBER AUTO_NEG
speed: 1000 Mbps now, 10000 Mbps max
1025(ae1): addr:ff:ff:ff:ff:ff:00
config: 0
state: LINK_UP
current: 1GB-FD COPPER
advertised: 1GB-FD 10GB-FD FIBER
supported: 10MB-FD 100MB-FD 1GB-FD 10GB-FD FIBER AUTO_NEG
speed: 1000 Mbps now, 10000 Mbps max
LOCAL(east): addr:0a:e6:1b:e4:40:40
config: 0
state: LINK_UP
current: 10MB-FD COPPER
supported: 10MB-FD COPPER
speed: 10 Mbps now, 10 Mbps max
OFPT_GET_CONFIG_REPLY (OF1.4) (xid=0x6): frags=normal miss_send_len=0
admin@PicOS-OVS$
Once the ports are configured and verified, flows can be managed in OVS.
Checking Flow Discrepancies
Check ovs-vswitchd flow discrepancies between the control plane and hardware:
admin@PicOS-OVS$ovs-ofctl dump-tables br0 | grep -v active=0: 0: active=4, lookup=n/a, matched=n/a admin@PicOS-OVS$ovs-ofctl dump-flows br0 OFPST_FLOW reply (OF1.4) (xid=0x2): cookie=0x0, duration=1449.903s, table=0, n_packets=n/a, n_bytes=0, in_port=1,dl_src=00:00:3d:a6:c8:f2 actions=output:2 cookie=0x0, duration=1444.537s, table=0, n_packets=n/a, n_bytes=0, in_port=1,dl_src=00:00:3d:a6:c9:14 actions=output:1 cookie=0x0, duration=71723.842s, table=0, n_packets=n/a, n_bytes=0, mpls,in_port=1,dl_vlan=1,mpls_label=10 actions=output:3 cookie=0x0, duration=74839.581s, table=0, n_packets=n/a, n_bytes=923443200, in_port=1 actions=output:2
Display hardware flows as shown below:
admin@PicOS-OVS$ovs-appctl pica/dump-flows #24 normal permanent priority=32769,in_port=1,dl_src=00:00:3d:a6:c8:f2, actions:2 #23 normal permanent priority=32769,in_port=1,dl_src=00:00:3d:a6:c9:14, actions:1 #22 normal permanent priority=32769,mpls,in_port=1,dl_vlan=1,mpls_label=10, actions:3 #21 normal permanent priority=32769,in_port=1, actions:2 #20 normal permanent priority=0, actions:drop Total 5 flows in HW.
Displaying OVSDB
Display the full OVSDB (Open vSwitch Database) as shown below:
admin@Leaf1$ovsdb-client dump
Bridge table
_uuid controller datapath_id datapath_type external_ids fail_mode flood_vlans flow_tables ipfix lldp_enable mirrors name netflow other_config ports protocols sflow status stp_enable
------------------------------------ -------------------------------------- ------------------ ------------- ------------ --------- ----------- ----------- ----- ----------- ------- -------- ------- ------------ ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -------------- ----- ------ ----------
c880536a-b614-41bf-9870-2d0bdab3664f [bedb4af7-2125-4346-8c89-bf61bd21f63b] "4c3e486e730203da" "pica8" {} [] [] {} [] false [] "ECODE3" [] {} [31605950-d9be-40b2-9ccb-bc4fd09991f0, 61ac5778-554f-4553-83ae-3bbc19ccf715, 62b35f47-e8ca-4496-8b37-f9bfbb7e80b0, 6dee5c6a-e9b8-41f7-87ef-b9379637a7c4, 99ac75b7-9fa1-4583-85f7-66d3145e7fa4] ["OpenFlow13"] [] {} false
<Some output omitted>
Debug Packet-In Messages
To debug the protocol messages between the switch and the controller, use the ovs-ofctl snoop command in the OVS mode. The following commands debug the protocol messages exchanged between the br0 bridge and the controller:
admin@Switch$ovs-ofctl snoop br0
Copyright © 2024 Pica8 Inc. All Rights Reserved.