Slow throughput (<100Mbps) between 2 Gbit Routers being used as switch

RouterOS Beginner Basics
1

This is a repost. I deleted my original post. I thought I made an error in the testing, but I did not.

Throughput test between 2 Powerbox Pro routers used as a switch (I plan to use them as outdoor Gbit POE switches).

Simple setup (no other config, zero firewall rules):

/interface bridge
add name=bridge1

/interface bridge port
add bridge=bridge1 interface=all

(POWERBOX PRO 1)
/ip address
add address=10.0.0.10/24 interface=bridge1 network=10.0.0.0

(POWERBOX PRO 2)
/ip address
add address=10.0.0.15/24 interface=bridge1 network=10.0.0.0


Throughput test:

tcp 2.png
The ether ports in both are showing ‘H’ for Hardware Offload (to switch chip). Why so slow? Why 100% CPU if the switch chip is supposedly doing the work?

Is it because the test is initiated from one one to the other, and therefore the test itself is loading the CPU?

What am I doing wrong.

2

Yes. Actually switching is not involved in your test case scenario … all traffic flows between CPUs of both powerboxes. To check switching capacity one should perform test using external equipment (e.g. two PCs running iperf or similar).

Note the text in first note in the bandwidth test documentation.

When CPU load gets high, one has to use tool->profile to see which process consumes the excess CPU.

3

I put a PBPro as a switch in-between a HAPAC2 (source) and another PBPro (target).

As you mentioned, the PBPro acting as a switch (middle) has low CPU @ 2%.

However since the speed test target is another PBPro, its CPU (@ 100%) is limiting the result, whereas the HAPAC2’s CPU hovers between 15-25%.

zz1.png
What I find interesting is the results are worse than the first test. The Rx throughput is only 56Mbps. I wonder if having a switch in the middle is reducing performance. Logically it shouldn’t, not at these speeds.

4

As long as CPU on devices running bwtest is at 100%, the test results are inconclusive as to how some box in the middle might affect the throughput.

5

So I put the Powerbox Pro (acting as switch) between a CCR1016-12G and a HAPAC2, and ran the bandwidth test again.

Simultaneous TCP Tx/Rx yields roughly 320Mbps/320Mbps. HAPAC2 @ 100% CPU, CCR at 10% CPU. Test limited by HAPAC2 CPU.
One way (CCR = test source) yields roughly 800Mbps (Rx) and 920Mbps (Tx). Similar CPU usage as above.

Powerbox Pro CPU% is consistently less than 3%, the switch chip offloading works.