PICOS OVS Troubleshooting

Owned by Tracy Yang (Unlicensed)
Apr 26, 2023

This section details basic procedures to troubleshoot PicOS switches in OVS (Open vSwitch) mode.

Verifying PicOS Mode

Verify if PicOS is actually running in OVS (Open vSwitch) mode, as described in Checking PicOS Mode.

When PicOS is running in the OVS mode, two processes should be running: ovsdb-server and ovs-vswitchd.

admin@XorPlus$ps -ef | grep ovs
root      1356     1  0 Jan26 ?        00:00:10 /ovs/sbin/ovsdb-server /ovs/ovs-vswitchd.conf.db --pidfile --remote=punix:/ovs/var/run/openvswitch/db.sock
root      1358     1  0 Jan26 ?        00:19:07 /ovs/sbin/ovs-vswitchd --enable-shared-lcmgr

In CrossFlow mode, the router stack must have been initialized in addition to having ovsdb-server and ovs-vswitchd processes running.

admin@XorPlus$ps -ef | grep pica
root     12430     1  0 Jan07 ?        00:05:49 pica_cardmgr
root     12432     1  0 Jan07 ?        01:03:19 pica_sif
root     12439     1  0 Jan07 ?        00:08:45 pica_lacp
root     12441     1 19 Jan07 ?        4-10:50:14 pica_lcmgr
root     12447     1  0 Jan07 ?        00:09:58 pica_login
root     13218     1  0 Jan07 ?        00:20:47 pica_mstp
root     13236     1  0 Jan07 ?        01:25:30 /pica/bin/xorp_rtrmgr -d -L local0.info -P /var/run/xorp_rtrmgr.pid

Verifying Bridge Configuration

For the bridge and ports to forward frames in hardware, the datapath_type configured for each entity must be set to pica8.

admin@PicOS-OVS$ovs-vsctl show
ac9e5b1e-4234-4158-9214-5660b9343779 
  Bridge east
      Controller "tcp:172.16.0.142:6653"
          is_connected: true
      fail_mode: standalone
      Port "ae1"
          tag: 1
          Interface "ae1"
              type: "pica8_lag"
              options: {lacp-mode=active, lacp-system-priority="32768", lacp-time=slow, lag_type=lacp, link_speed=auto, members="te-1/1/2"}
      Port "te-1/1/2"
          tag: 1
          Interface "te-1/1/2"
              type: "pica8"
              options: {flow_ctl=none, link_speed=auto}
      Port "te-1/1/1"
          tag: 1
          Interface "te-1/1/1"
              type: "pica8"
              options: {flow_ctl=none, link_speed=auto}

admin@PicOS-OVS$ovs-ofctl show east
OFPT_FEATURES_REPLY (OF1.4) (xid=0x2): dpid:1deb0ae61be44040
n_tables:254, n_buffers:256
capabilities: FLOW_STATS TABLE_STATS PORT_STATS GROUP_STATS
OFPST_PORT_DESC reply (OF1.4) (xid=0x4):
 1(te-1/1/1): addr:ff:ff:ff:ff:ff:00
     config:     0
     state:      LINK_UP
     current:    1GB-FD COPPER
     advertised: 1GB-FD 10GB-FD FIBER
     supported:  10MB-FD 100MB-FD 1GB-FD 10GB-FD FIBER AUTO_NEG
     speed: 1000 Mbps now, 10000 Mbps max
 2(te-1/1/2): addr:ff:ff:ff:ff:ff:00
     config:     0
     state:      LINK_DOWN
     current:    1GB-FD COPPER
     advertised: 1GB-FD 10GB-FD FIBER
     supported:  10MB-FD 100MB-FD 1GB-FD 10GB-FD FIBER AUTO_NEG
     speed: 1000 Mbps now, 10000 Mbps max
 1025(ae1): addr:ff:ff:ff:ff:ff:00
     config:     0
     state:      LINK_UP
     current:    1GB-FD COPPER
     advertised: 1GB-FD 10GB-FD FIBER
     supported:  10MB-FD 100MB-FD 1GB-FD 10GB-FD FIBER AUTO_NEG
     speed: 1000 Mbps now, 10000 Mbps max
 LOCAL(east): addr:0a:e6:1b:e4:40:40
     config:     0
     state:      LINK_UP
     current:    10MB-FD COPPER
     supported:  10MB-FD COPPER
     speed: 10 Mbps now, 10 Mbps max
OFPT_GET_CONFIG_REPLY (OF1.4) (xid=0x6): frags=normal miss_send_len=0
admin@PicOS-OVS$

Once the ports are configured and verified, flows can be managed in OVS.

Checking Flow Discrepancies

Check ovs-vswitchd flow discrepancies between the control plane and hardware:

admin@PicOS-OVS$ovs-ofctl dump-tables br0 | grep -v active=0:
  0: active=4, lookup=n/a, matched=n/a

admin@PicOS-OVS$ovs-ofctl dump-flows br0
OFPST_FLOW reply (OF1.4) (xid=0x2):
 cookie=0x0, duration=1449.903s, table=0, n_packets=n/a, n_bytes=0, in_port=1,dl_src=00:00:3d:a6:c8:f2 actions=output:2
 cookie=0x0, duration=1444.537s, table=0, n_packets=n/a, n_bytes=0, in_port=1,dl_src=00:00:3d:a6:c9:14 actions=output:1
 cookie=0x0, duration=71723.842s, table=0, n_packets=n/a, n_bytes=0, mpls,in_port=1,dl_vlan=1,mpls_label=10 actions=output:3
 cookie=0x0, duration=74839.581s, table=0, n_packets=n/a, n_bytes=923443200, in_port=1 actions=output:2

Display hardware flows as shown below:

admin@PicOS-OVS$ovs-appctl pica/dump-flows
#24 normal permanent priority=32769,in_port=1,dl_src=00:00:3d:a6:c8:f2, actions:2
#23 normal permanent priority=32769,in_port=1,dl_src=00:00:3d:a6:c9:14, actions:1
#22 normal permanent priority=32769,mpls,in_port=1,dl_vlan=1,mpls_label=10, actions:3
#21 normal permanent priority=32769,in_port=1, actions:2
#20 normal permanent priority=0, actions:drop
Total 5 flows in HW.

Displaying OVSDB

Display the full OVSDB (Open vSwitch Database) as shown below:

admin@Leaf1$ovsdb-client dump
Bridge table
_uuid                                controller                             datapath_id        datapath_type external_ids fail_mode flood_vlans flow_tables ipfix lldp_enable mirrors name     netflow other_config ports                                                                                                                                                                                          protocols      sflow status stp_enable
------------------------------------ -------------------------------------- ------------------ ------------- ------------ --------- ----------- ----------- ----- ----------- ------- -------- ------- ------------ ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -------------- ----- ------ ----------
c880536a-b614-41bf-9870-2d0bdab3664f [bedb4af7-2125-4346-8c89-bf61bd21f63b] "4c3e486e730203da" "pica8"       {}           []        []          {}          []    false       []      "ECODE3" []      {}           [31605950-d9be-40b2-9ccb-bc4fd09991f0, 61ac5778-554f-4553-83ae-3bbc19ccf715, 62b35f47-e8ca-4496-8b37-f9bfbb7e80b0, 6dee5c6a-e9b8-41f7-87ef-b9379637a7c4, 99ac75b7-9fa1-4583-85f7-66d3145e7fa4] ["OpenFlow13"] []    {}     false     
<Some output omitted>

Debug Packet-In Messages

 

To debug the protocol messages between the switch and the controller, use the ovs-ofctl snoop command in the OVS mode. The following commands debug the protocol messages exchanged between the br0 bridge and the controller:

 

admin@Switch$ovs-ofctl snoop br0



Copyright © 2024 Pica8 Inc. All Rights Reserved.