I added a 4-port 802.11ad bonding interface to our new CCR2004-16G-2S+ router as a connection to the main switch (its SFP+ slots are already occupied so I cannot use a 10G DAC cable).
I wonder if this configuration will be hw-offloaded. There is a comment in the manual that the bonding interface is hw accelerated as member of a bridge, but in this case the interface is not member of a bridge, it is a plain interface towards the local network connected to an external switch (using 4 cables).
Is being member of a bridge necessary for the bonding function to be offloaded? Would it be worthwhile to setup a dummy bridge just for that?
From various discussions around the forum (andvreading a few MT manual pages) I drew conclusion that only bridge functions are offloaded to switch chips. And nothing else. Unless we’re talking about models with switch chip config exposed through ROS UI, I guess CCR2004 isn’t one of them.
In your case this means that bonding is only offloaded to switch if bond interface is made bridge port and that particular bridge is offloaded. And then one may hit the limitation that only single bridge can be offloaded (to one switch chip).
So if your CCR doesn’t currently run any bridge, it might be benefitial to insert a bridge between bond interface and software. This should be pretty straight forward, so you might first do it without bridge, check performance (both bond throughput and CPU load), and tgen with bridge (again checking performance). And report back the results
Well, at this moment I can still change the configuration, but I cannot experiment with this router once I have put it into production.
(i.e. try both alternatives)
And before it is in production, it is difficult to do any performance testing. Only thing I can think of is temporarily make a bonding interface on the other switch as well, cross-wire them, and run a bandwidth test over that. But, my experience is that bandwidth test on MikroTik routers uses much more CPU than the actual routing one is trying to test (hence the advise to use external devices for the bandwith test)…
The device is purely a router, there aren’t any bridged ports. The CCR2004 has a switch menu but no configuration options for 802.11ad in there.
If I understand you (and the docs) correctly, when I put in a bridge with only that bonding interface as a port, and have this port setup with hw=yes, it will offload the bonding to the switch chip. And when I just make the bonding interface and directly use it for routing, it will not.
I see indeed that when I put the bonding interface in a bridge, and enable hw offload on the port, it becomes hw offloaded.
The is no real insight in what is exactly done where, but I assume it will work correctly.
Did not see any benefit of offloading a single bonding interface (through a bridge) vs stand-alone bonding.
The only benefit you will get is that switch is doing the transmit hash for Tx and collecting Rx packets. But you add a bridge layer, and every packet goes through the software bridge input/output chains, which seems to be more CPU intensive than letting it do the load balancing for bonding.
So, adding a bridge with a single bridge port to get the “H” flag will not result in any throughput increase or lowered CPU load. You might actually be doing the opposite.
I lost around 10% throughput in a basic routing setup when using CCR2004-16G-2S+. The bridge HW offloading makes more sense when you use multiple HW offloaded ports.
Thanks for that information! For us customers without elaborate test labs it is difficult to test this.
So it could be a good idea to put that in the documentation…
That’s how we do it for CCR2116 are you saying that it doesn’t have real benefits? we just create a bridge for this purpose and put bonding interface on it sad can you made it clear in the docs please
What are you expecting to offload and what was your motivation to make this configuration, @loloski?
There can be benefits in some cases, for example, if your device is L3HW capable and you wish to offload inter-VLAN routing on bond. For that to work, you need HW offloaded vlan-filtering bridge.
Edit:
Of course, the main benefit is when HW offloading on the bond is used together with other HW offloaded ports (single HW bridge with bond and other bonds/Ethernet).
At least what I expected was that the hw offload would do the hashing and selection of interface in hardware.
I.e. “the router” sends the packets to the switch chip as if it is a single interface and the switch chip does the entire bonding thing.
I cannot judge how much benefit that brings, and if it is offset by having a bridge with one port (with fast-forward) in order to implement it in RouterOS.
As I had already made the config just before your reply above that it would not be better, I first took the router to production with that solution.
It was a VLAN-filtering bridge with 5 VLAN interfaces connected to it, and the bonding interface as its only port. The hw accel was enabled.
However, it turns out it does not work right! I made SUP-122465 about it.
So I removed the bridge and connected the VLAN interfaces directly to the bonding interface instead. Now it looks like it works right.
It is difficult to see any CPU usage difference at the moment.
sad can you made it clear in the docs please 