CRS112 uplink high tx-drop-packet and tx-drop-byte

Hi,

  • hardware: CRS112-8G-4S
  • RouterOS: 7.22

The switch is really just switching and the CPU load is low. There are six servers behind it (black boxes for me), and ISP uplink connected. All 1Gbps Ethernet.

I am seeing a very high number of tx-drop-packet and tx-drop-byte on my uplink interface. No other kinds of errors.

I am trying to understand what’s happening, but can’t actually find any clear definition of tx-drop-packet and tx-drop-byte in the docs. I suppose that the relevant line in the ROS → Wired connections → Ethernet docs is:

tx-drop (integer) Total count of transmitted frames that were dropped due to the already full output queue

Is this correct? It seems to me this contains the drops across all the queues. There are Tx counters per queues, but the drop counters per queue are missing.

Interestingly, when I run /interface monitor-traffic ether7uplink, the tx-queue-drops-per-secondnever shows anything but 0, and the tx-bits-per-second is in higher tens or lower hundreds (mostly 50 - 200 mbps, although now I see a few bursts to about 800 mbps when I look at the WinBox graph). The tx-drop-* grows even in these 30-200 Mbps periods, outside of the visible 800Mbps spikes.

What I would like to understand is:

  1. Is this due to bursts leading to queue buffer overflow that are not visible on the graph?
  2. If so, why on the Traffic tab, why Tx/Rx Drops and Tx Queue Drops are 0?
  3. Is this hitting packet per second or bits per second ceiling?
  4. Can something be done about this problem? ISP say they don’t see any errors and don’t do any traffic shaping on this port.
    1. I briefly switched the port from only-hardware-queue to multi-queue-ethernet (mq pfifo, queue size 50 - the default), seemed make no difference.

Looking forward to learning more about this!:slight_smile:

This is a few minutes after resetting the counters:

> /interface ethernet print stats where name=ether7uplink
                            ;;;                     uplink
                          name:               ether7uplink
                driver-rx-byte:                  9 411 093
              driver-rx-packet:                     94 204
                driver-tx-byte:                 43 463 782
              driver-tx-packet:                     39 833
                      rx-bytes:              9 009 226 292
                  rx-too-short:                          0
                         rx-64:                     55 118
                     rx-65-127:                 21 323 373
                    rx-128-255:                  2 819 468
                    rx-256-511:                  5 682 842
                   rx-512-1023:                  1 205 103
                  rx-1024-1518:                  2 418 144
                   rx-1519-max:                          0
                   rx-too-long:                          0
                  rx-broadcast:                     54 188
                      rx-pause:                          0
                  rx-multicast:                     12 655
                  rx-fcs-error:                          0
                rx-align-error:                          0
                   rx-fragment:                          0
                   rx-overflow:                          0
                      tx-bytes:             25 340 437 941
                         tx-64:                      1 190
                     tx-65-127:                 18 449 007
                    tx-128-255:                  2 051 444
                    tx-256-511:                  6 801 873
                   tx-512-1023:                  1 271 487
                  tx-1024-1518:                 13 014 507
                   tx-1519-max:                          0
                   tx-too-long:                          0
                  tx-broadcast:                      1 128
                      tx-pause:                          0
                  tx-multicast:                     15 804
                   tx-underrun:                          0
                  tx-collision:                          0
        tx-excessive-collision:                          0
         tx-multiple-collision:                          0
           tx-single-collision:                          0
         tx-excessive-deferred:                          0
                   tx-deferred:                          0
             tx-late-collision:                          0
              tx-queue0-packet:                      3 440
              tx-queue1-packet:                 41 585 668
              tx-queue2-packet:                        388
              tx-queue3-packet:                         12
              tx-queue4-packet:                          0
              tx-queue5-packet:                          0
              tx-queue6-packet:                          0
              tx-queue7-packet:                          0
                tx-queue0-byte:                    272 048
                tx-queue1-byte:             25 340 133 505
                tx-queue2-byte:                     31 260
                tx-queue3-byte:                      1 128
                tx-queue4-byte:                          0
                tx-queue5-byte:                          0
                tx-queue6-byte:                          0
                tx-queue7-byte:                          0
                tx-drop-packet:                     87 394
                  tx-drop-byte:                115 580 817
  tx-queue-custom0-drop-packet:                          0
    tx-queue-custom0-drop-byte:                          0
  tx-queue-custom1-drop-packet:                          0
    tx-queue-custom1-drop-byte:                          0
            policy-drop-packet:                          0
            custom-drop-packet:                          0
               current-learned: 18 446 744 073 709 551 612
                   not-learned:                          0


More details:

> /system resource print
                   uptime: 5d16h7m34s         
                  version: 7.22 (stable)      
               build-time: 2026-03-09 08:38:02
         factory-software: 6.44.6             
              free-memory: 74.6MiB            
             total-memory: 128.0MiB           
                      cpu: MIPS 24Kc V8.5     
                cpu-count: 1                  
            cpu-frequency: 400MHz             
                 cpu-load: 46%                
           free-hdd-space: 2968.0KiB          
          total-hdd-space: 16.0MiB            
  write-sect-since-reboot: 661                
         write-sect-total: 3362018            
        architecture-name: mipsbe             
               board-name: CRS112-8G-4S       
                 platform: MikroTik

> /system routerboard print
       routerboard: yes         
             model: CRS112-8G-4S
          revision: r3          
     serial-number: HE108JN1WA6 
     firmware-type: qca8513L    
  factory-firmware: 6.48.6      
  current-firmware: 7.22        
  upgrade-firmware: 7.22 

> /interface ethernet print detail where name=ether7uplink
Flags: X - DISABLED, R - RUNNING; S - SLAVE 
 0 RS ;;; uplink
      name="ether7uplink" default-name="ether7" mtu=1500 l2mtu=1588 mac-address=48:A9:8A:35:03:C5 orig-mac-address=48:A9:8A:35:03:C5 arp=enabled arp-timeout=auto loop-protect=default loop-protect-status=off 
      loop-protect-send-interval=5s loop-protect-disable-time=5m auto-negotiation=yes advertise=10M-baseT-half,10M-baseT-full,100M-baseT-half,100M-baseT-full,1G-baseT-half,1G-baseT-full tx-flow-control=off 
      rx-flow-control=off bandwidth=unlimited/unlimited switch=switch1

> /interface bridge print detail
Flags: Y - MANAGED; D - DYNAMIC; X - DISABLED, R - RUNNING 
 0   R ;;; defconf
       name="bridge" mtu=auto actual-mtu=1500 l2mtu=1588 arp=enabled arp-timeout=auto mac-address=48:A9:8A:35:03:BF protocol-mode=none fast-forward=yes igmp-snooping=no auto-mac=no admin-mac=48:A9:8A:35:03:BF 
       ageing-time=5m vlan-filtering=no dhcp-snooping=no ra-guard=no port-cost-mode=short forward-reserved-addresses=no max-learned-entries=auto mlag-peer-port=none mlag-priority=128 mlag-heartbeat=5s

> /interface bridge port print detail
...
 6   H  ;;; defconf
        interface=ether7uplink bridge=bridge priority=0x80 path-cost=10 internal-path-cost=10 edge=auto point-to-point=auto learn=auto horizon=none hw=yes auto-isolate=no restricted-role=no restricted-tcn=no pvid=1 
        frame-types=admit-all ingress-filtering=no unknown-unicast-flood=yes unknown-multicast-flood=yes broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no trusted-ra=no mvrp-registrar-state=normal 
        mvrp-applicant-state=normal-participant multicast-router=temporary-query fast-leave=no

> /interface ethernet switch port print detail

 6   name="ether7uplink" ingress-customer-tpid=0x8100 egress-customer-tpid=0x8100 ingress-service-tpid=0x88A8 egress-service-tpid=0x88A8 learn=yes drop-secure-static-mac-move=no drop-dynamic-mac-move=no 
     allow-unicast-loopback=no allow-multicast-loopback=no action-on-static-station-move=forward drop-when-ufdb-entry-src-drop=yes isolation-leakage-profile=29 vlan-type=network-port allow-fdb-based-vlan-translate=no 
     allow-mac-based-service-vlan-assignment-for=all allow-mac-based-customer-vlan-assignment-for=all filter-untagged-frame=no filter-priority-tagged-frame=no filter-tagged-frame=no 
     egress-vlan-tag-table-lookup-key=egress-vid egress-vlan-mode=unmodified ingress-mirror-to=none ingress-mirroring-according-to-vlan=no egress-mirror-to=none 
     qos-scheme-precedence=ingress-acl-based,sa-based,da-based,dscp-based,protocol-based,vlan-based,pcp-based default-customer-pcp=0 default-service-pcp=0 pcp-propagation-for-initial-pcp=no egress-pcp-propagation=no 
     dscp-based-qos-dscp-to-dscp-mapping=yes pcp-or-dscp-based-qos-change-dei=no pcp-or-dscp-based-qos-change-pcp=no pcp-or-dscp-based-qos-change-dscp=no pcp-based-qos-drop-precedence-mapping=0-15:green 
     pcp-based-qos-dscp-mapping=0-15:0 pcp-based-qos-dei-mapping=0-15:0 pcp-based-qos-pcp-mapping=0-15:0 pcp-based-qos-priority-mapping=0-15:0 priority-to-queue=0-15:0,1:1,2:2,3:3 
     per-queue-scheduling=wrr-group0:1,wrr-group0:2,wrr-group0:4,wrr-group0:8,wrr-group0:16,wrr-group0:32,wrr-group0:64,wrr-group0:128 custom-drop-counter-includes="" queue-custom-drop-counter0-includes="" 
     queue-custom-drop-counter1-includes="" policy-drop-counter-includes="" 

> /interface ethernet switch print       
                                                     name: switch1                        
                                                     type: QCA-8511                       
                                              bridge-type: customer-vid-used-as-lookup-vid
                      drop-if-no-vlan-assignment-on-ports:                                
  drop-if-invalid-or-src-port-not-member-of-vlan-on-ports:                                
                                 unknown-vlan-lookup-mode: svl                            
                                     forward-unknown-vlan: yes                            
                          use-svid-in-one2one-vlan-lookup: no                             
                          use-cvid-in-one2one-vlan-lookup: yes                            
                                      mac-level-isolation: yes                            
                                    multicast-lookup-mode: dst-ip-and-vid-for-ipv4        
                         override-existing-when-ufdb-full: no                             
                                      unicast-fdb-timeout: 5m                             
                                          ingress-mirror0: switch1-cpu,unmodified         
                                          ingress-mirror1: switch1-cpu,unmodified         
                                     ingress-mirror-ratio: 1/1                            
                                           egress-mirror0: switch1-cpu,modified           
                                           egress-mirror1: switch1-cpu,modified           
                                      egress-mirror-ratio: 1/1                            
                                                 fdb-uses: mirror0                        
                                                vlan-uses: mirror0                        
                        mirror-egress-if-ingress-mirrored: no                             
                                 mirror-tx-on-mirror-port: no                             
                             mirrored-packet-qos-priority: 0                              
                          mirrored-packet-drop-precedence: green                          
                           bypass-vlan-ingress-filter-for:                                
                         bypass-ingress-port-policing-for:                                
                      bypass-l2-security-check-filter-for: 

You could maybe try multi-queue-ethernet-default as per the following topic.

Thanks for the reply @rplant!

I saw exactly this topic and already tried that - the counter continued to grow (I didn't, however, notice/graph if it grew more slowly or not).

You could try enabling tx flow control and rx flow control on all interfaces,
and see if that helps. (And doesn't break other things.)

I have exacly the same issue Mikrotik CRS318-16P-2S+OUT - with 7.16 and 7.19 this issue does not exist. If we upgrade a old Version to the new 7.22 the issue starts.

We find out that this issue currupt IPSec Tunnels and traffic in this Tunnels. For example in some situation RDP Connections drop in the tunnel, while ping traffic works without issues or Voip calls drops.

vxlans over this interfaces has issues too. To play with other queue types or sizes doesnt change anything.

Enable/Disable Flow Control does not have a effect of this issue.