Network card problem

1

When starting the server version 5.5.0.3, I get an error:

Mar 12 16:01:10 name asd[9409]: Mar 12 2021 16:01:10 GMT: CRITICAL (hardware): (hardware.c:1720) interface eth2 with inconsistent number of RX interrupts: 8 vs. 4

Config:

network {
  service {
    address aggi
    access-address aggi
    # alternate-access-address y.y.y.y
    port 3000
  }

  heartbeat {
    address aggi
    port 9918
    interval 150
    timeout 10

    mode mesh
    mesh-seed-address-port host1.name.net 9918
    mesh-seed-address-port host2.name.net 9918
    mesh-seed-address-port host3.name.net 9918
  }

  fabric {
    address aggi
    port 3001
  }

  info {
    address aggi
    port 3003
  }
}

Ifaces

agge: flags=5187<UP,BROADCAST,RUNNING,MASTER,MULTICAST>  mtu 1500
        inet a.b.c.d  netmask 255.255.255.248  broadcast a.b.c.d
        inet6 fe80::2eea:7fff:fedb:91e7  prefixlen 64  scopeid 0x20<link>
        ether 2c:ea:7f:db:91:e7  txqueuelen 1000  (Ethernet)
        RX packets 121360  bytes 79937701 (79.9 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 7030  bytes 598390 (598.3 KB)
        TX errors 0  dropped 3 overruns 0  carrier 0  collisions 0

aggi: flags=5187<UP,BROADCAST,RUNNING,MASTER,MULTICAST>  mtu 1500
        inet a.b.c.d  netmask 255.255.255.248  broadcast a.b.c.d
        inet6 fe80::2eea:7fff:fedb:91e6  prefixlen 64  scopeid 0x20<link>
        ether 2c:ea:7f:db:91:e6  txqueuelen 1000  (Ethernet)
        RX packets 131796  bytes 22166229 (22.1 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 105686  bytes 49012918 (49.0 MB)
        TX errors 0  dropped 2 overruns 0  carrier 0  collisions 0

eth0: flags=6211<UP,BROADCAST,RUNNING,SLAVE,MULTICAST>  mtu 1500
        ether 2c:ea:7f:db:91:e6  txqueuelen 1000  (Ethernet)
        RX packets 1132  bytes 144600 (144.6 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 63045  bytes 40720259 (40.7 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 17

eth1: flags=6211<UP,BROADCAST,RUNNING,SLAVE,MULTICAST>  mtu 1500
        ether 2c:ea:7f:db:91:e7  txqueuelen 1000  (Ethernet)
        RX packets 121360  bytes 79937701 (79.9 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 7030  bytes 598390 (598.3 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 18

eth2: flags=6211<UP,BROADCAST,RUNNING,SLAVE,MULTICAST>  mtu 1500
        ether 2c:ea:7f:db:91:e6  txqueuelen 1000  (Ethernet)
        RX packets 130664  bytes 22021629 (22.0 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 42641  bytes 8292659 (8.2 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device memory 0x7bb00000-7bbfffff

eth3: flags=6147<UP,BROADCAST,SLAVE,MULTICAST>  mtu 1500
        ether 2c:ea:7f:db:91:e7  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device memory 0x7ba00000-7bafffff

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 3858  bytes 209601 (209.6 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 3858  bytes 209601 (209.6 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

I don’t see any problems with the network. Which way to look or what can be done?

2

There seems to be some issue with number of network queues for that interface. Which network interface model is this? OS?

The aggi seems to be a bonded interface(eth0,eth2). Are each member interface identical in terms of network queues?

What’s the output for :

cat /proc/interrupts

and

ethtool -l eth0

ethtool -l eth1

ethtool -l eth2

ethtool -l eth3

3

Thank you for answer. See below for more information:

Ubuntu Server 20.04

04:00.0 Ethernet controller: Broadcom Inc. and subsidiaries NetXtreme BCM5720 Gigabit Ethernet PCIe
04:00.1 Ethernet controller: Broadcom Inc. and subsidiaries NetXtreme BCM5720 Gigabit Ethernet PCIe
3b:00.0 Ethernet controller: Intel Corporation Ethernet Controller 10G X550T (rev 01)
3b:00.1 Ethernet controller: Intel Corporation Ethernet Controller 10G X550T (rev 01)
af:00.0 Ethernet controller: Intel Corporation Ethernet Controller 10G X550T (rev 01)
af:00.1 Ethernet controller: Intel Corporation Ethernet Controller 10G X550T (rev 01)

$ ethtool -l eth0
Channel parameters for eth0:
Pre-set maximums:
RX:		4
TX:		4
Other:		0
Combined:	0
Current hardware settings:
RX:		4
TX:		4
Other:		0
Combined:	0

$ ethtool -l eth1
Channel parameters for eth1:
Pre-set maximums:
RX:		4
TX:		4
Other:		0
Combined:	0
Current hardware settings:
RX:		4
TX:		4
Other:		0
Combined:	0

$ ethtool -l eth2
Channel parameters for eth2:
Pre-set maximums:
RX:		0
TX:		0
Other:		1
Combined:	63
Current hardware settings:
RX:		0
TX:		0
Other:		1
Combined:	40

$ ethtool -l eth3
Channel parameters for eth3:
Pre-set maximums:
RX:		0
TX:		0
Other:		1
Combined:	63
Current hardware settings:
RX:		0
TX:		0
Other:		1
Combined:	40

/proc/interrupts is very large, does not fit into the message.

4

Does eth2 driver allow for setting of RX and TX via ethtool? Seems the bonding of network cards with different network queues or unset RX and TX queues could be at play here. Wondering if ethtool will let you set RX and TX on eth2 to 4 and match eth0. Or not use aggi and try to reconfigure to use eth2 with no bonding…

ethtool -L|--set-channels devname [rx N] [tx N] [other N]
          [combined N]
5

We cannot refuse aggi, we need an interface with a bandwidth of 2Gbps. The cards themselves do not allow you to change anything. Yes, this is strange, we deal with it. The question is - can this AS behavior be changed?

6

Can you try capturing the following:

ls /sys/class/net/*/queues/rx-*/rps_cpus |cut -d '/' -f 5|sort|uniq -c
ls /sys/class/net/*/queues/tx-*/xps_cpus |cut -d '/' -f 5|sort|uniq -c

This should give us the actual queue counts.

7

Unfortunately, I no longer have the opportunity to check. They asked to install another card. If the opportunity to check arises, I will definitely check it.

8

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.