PICOS OVS Troubleshooting
This section details basic procedures to troubleshoot PicOS switches in OVS (Open vSwitch) mode.
Verifying PicOS Mode
Verify if PicOS is actually running in OVS (Open vSwitch) mode, as described in Checking PicOS Mode.
When PicOS is running in the OVS mode, two processes should be running: ovsdb-server and ovs-vswitchd.
admin@XorPlus$ps -ef | grep ovs root 1356 1 0 Jan26 ? 00:00:10 /ovs/sbin/ovsdb-server /ovs/ovs-vswitchd.conf.db --pidfile --remote=punix:/ovs/var/run/openvswitch/db.sock root 1358 1 0 Jan26 ? 00:19:07 /ovs/sbin/ovs-vswitchd --enable-shared-lcmgr
In CrossFlow mode, the router stack must have been initialized in addition to having ovsdb-server and ovs-vswitchd processes running.
admin@XorPlus$ps -ef | grep pica root 12430 1 0 Jan07 ? 00:05:49 pica_cardmgr root 12432 1 0 Jan07 ? 01:03:19 pica_sif root 12439 1 0 Jan07 ? 00:08:45 pica_lacp root 12441 1 19 Jan07 ? 4-10:50:14 pica_lcmgr root 12447 1 0 Jan07 ? 00:09:58 pica_login root 13218 1 0 Jan07 ? 00:20:47 pica_mstp root 13236 1 0 Jan07 ? 01:25:30 /pica/bin/xorp_rtrmgr -d -L local0.info -P /var/run/xorp_rtrmgr.pid
Verifying Bridge Configuration
For the bridge and ports to forward frames in hardware, the datapath_type configured for each entity must be set to pica8.
admin@PicOS-OVS$ovs-vsctl show ac9e5b1e-4234-4158-9214-5660b9343779 Bridge east Controller "tcp:172.16.0.142:6653" is_connected: true fail_mode: standalone Port "ae1" tag: 1 Interface "ae1" type: "pica8_lag" options: {lacp-mode=active, lacp-system-priority="32768", lacp-time=slow, lag_type=lacp, link_speed=auto, members="te-1/1/2"} Port "te-1/1/2" tag: 1 Interface "te-1/1/2" type: "pica8" options: {flow_ctl=none, link_speed=auto} Port "te-1/1/1" tag: 1 Interface "te-1/1/1" type: "pica8" options: {flow_ctl=none, link_speed=auto} admin@PicOS-OVS$ovs-ofctl show east OFPT_FEATURES_REPLY (OF1.4) (xid=0x2): dpid:1deb0ae61be44040 n_tables:254, n_buffers:256 capabilities: FLOW_STATS TABLE_STATS PORT_STATS GROUP_STATS OFPST_PORT_DESC reply (OF1.4) (xid=0x4): 1(te-1/1/1): addr:ff:ff:ff:ff:ff:00 config: 0 state: LINK_UP current: 1GB-FD COPPER advertised: 1GB-FD 10GB-FD FIBER supported: 10MB-FD 100MB-FD 1GB-FD 10GB-FD FIBER AUTO_NEG speed: 1000 Mbps now, 10000 Mbps max 2(te-1/1/2): addr:ff:ff:ff:ff:ff:00 config: 0 state: LINK_DOWN current: 1GB-FD COPPER advertised: 1GB-FD 10GB-FD FIBER supported: 10MB-FD 100MB-FD 1GB-FD 10GB-FD FIBER AUTO_NEG speed: 1000 Mbps now, 10000 Mbps max 1025(ae1): addr:ff:ff:ff:ff:ff:00 config: 0 state: LINK_UP current: 1GB-FD COPPER advertised: 1GB-FD 10GB-FD FIBER supported: 10MB-FD 100MB-FD 1GB-FD 10GB-FD FIBER AUTO_NEG speed: 1000 Mbps now, 10000 Mbps max LOCAL(east): addr:0a:e6:1b:e4:40:40 config: 0 state: LINK_UP current: 10MB-FD COPPER supported: 10MB-FD COPPER speed: 10 Mbps now, 10 Mbps max OFPT_GET_CONFIG_REPLY (OF1.4) (xid=0x6): frags=normal miss_send_len=0 admin@PicOS-OVS$
Once the ports are configured and verified, flows can be managed in OVS.
Checking Flow Discrepancies
Check ovs-vswitchd flow discrepancies between the control plane and hardware:
admin@PicOS-OVS$ovs-ofctl dump-tables br0 | grep -v active=0: 0: active=4, lookup=n/a, matched=n/a admin@PicOS-OVS$ovs-ofctl dump-flows br0 OFPST_FLOW reply (OF1.4) (xid=0x2): cookie=0x0, duration=1449.903s, table=0, n_packets=n/a, n_bytes=0, in_port=1,dl_src=00:00:3d:a6:c8:f2 actions=output:2 cookie=0x0, duration=1444.537s, table=0, n_packets=n/a, n_bytes=0, in_port=1,dl_src=00:00:3d:a6:c9:14 actions=output:1 cookie=0x0, duration=71723.842s, table=0, n_packets=n/a, n_bytes=0, mpls,in_port=1,dl_vlan=1,mpls_label=10 actions=output:3 cookie=0x0, duration=74839.581s, table=0, n_packets=n/a, n_bytes=923443200, in_port=1 actions=output:2
Display hardware flows as shown below:
admin@PicOS-OVS$ovs-appctl pica/dump-flows #24 normal permanent priority=32769,in_port=1,dl_src=00:00:3d:a6:c8:f2, actions:2 #23 normal permanent priority=32769,in_port=1,dl_src=00:00:3d:a6:c9:14, actions:1 #22 normal permanent priority=32769,mpls,in_port=1,dl_vlan=1,mpls_label=10, actions:3 #21 normal permanent priority=32769,in_port=1, actions:2 #20 normal permanent priority=0, actions:drop Total 5 flows in HW.
Displaying OVSDB
Display the full OVSDB (Open vSwitch Database) as shown below:
admin@Leaf1$ovsdb-client dump Bridge table _uuid controller datapath_id datapath_type external_ids fail_mode flood_vlans flow_tables ipfix lldp_enable mirrors name netflow other_config ports protocols sflow status stp_enable ------------------------------------ -------------------------------------- ------------------ ------------- ------------ --------- ----------- ----------- ----- ----------- ------- -------- ------- ------------ ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -------------- ----- ------ ---------- c880536a-b614-41bf-9870-2d0bdab3664f [bedb4af7-2125-4346-8c89-bf61bd21f63b] "4c3e486e730203da" "pica8" {} [] [] {} [] false [] "ECODE3" [] {} [31605950-d9be-40b2-9ccb-bc4fd09991f0, 61ac5778-554f-4553-83ae-3bbc19ccf715, 62b35f47-e8ca-4496-8b37-f9bfbb7e80b0, 6dee5c6a-e9b8-41f7-87ef-b9379637a7c4, 99ac75b7-9fa1-4583-85f7-66d3145e7fa4] ["OpenFlow13"] [] {} false <Some output omitted>
Debug Packet-In Messages
To debug the protocol messages between the switch and the controller, use the ovs-ofctl snoop command in the OVS mode. The following commands debug the protocol messages exchanged between the br0 bridge and the controller:
admin@Switch$ovs-ofctl snoop br0
Copyright © 2024 Pica8 Inc. All Rights Reserved.