How exactly are you testing? L2 HW offload only works for traffic within same VLAN between different bridge ports. Your config has only a pair of pirts belonging to same VLAN (e.g. sfp-sfpplus 1 and 3 in VID 66 etc.). So make sure tests are done correctly.
For traffic between different VLANs you need router. CRS3xx can offload L3 to HW, in that case it can route wirespeed. The config shown doesn’t show L3 setup, so it’s impossible to say anything about it.
If L3 setup doesn’t exist on this CRS, then the routing between VLANs is done by another device (router) and performance bottleneck is there to be found. And even if L3 setup is present, it doesn’t mean this device will do the routing, it all depends on setup of all involved devices (servers, main router, …)
Yes, I see 100% CPU load when I am pushing 800Mbit between two ports.
I have a device plugged into sfp-sfpplus 1 which is setup to send tagged traffic on VLAN 66 and 67, and another device plugged into sfp-sfpplus 3 which is setup to send tagged traffic on vlan’s 66 and 68. I am using iperf on both of these devices.
For traffic between different VLANs you need router. CRS3xx can offload L3 to HW, in that case it can route wirespeed. The config shown doesn’t show L3 setup, so it’s impossible to say anything about it.
If L3 setup doesn’t exist on this CRS, then the routing between VLANs is done by another device (router) and performance bottleneck is there to be found. And even if L3 setup is present, it doesn’t mean this device will do the routing, it all depends on setup of all involved devices (servers, main router, …)
I am not routing traffic between VLANs, I am using VLAN’s for isolation only. Hence my expectation that this would all be done in the switch chip of the CRS305, and not hit the CPU at all, thus giving me near wire speed performance. This should all be happening at L2 only.
My hope is that I would have only tagged traffic on this switch, there should be no untagged traffic. The PVID is only configured as it seems to be a requirement to do so. Hence why it is different from the VLAN ID.
In which case you do not need any /interface vlan entries - these provide the link between tagged VLANs in the bridge and services on the Mikrotik itself, via the implicit bridge-to-CPU port. These, plus the bridge ports having tagged=External-Bridge,… will direct any unicast to unknown MAC addresses plus broadcast and multicast traffic in these VLANs to the CPU and require processing.
My hope is that I would have only tagged traffic on this switch, there should be no untagged traffic. The PVID is only configured as it seems to be a requirement to do so. Hence why it is different from the VLAN ID.
The pvid= setting always exists and defaults to 1, if you set frame-types=admit-only-vlan-tagged untagged membership is disabled and the PVID setting ignored.
(Everyone who keeps mentioning Layer 3 isn’t reading the thread very well.)
Have you checked to see what speeds the ports are negotiating with the hosts? Are both ends (switch and host) reporting 10Gbps?
Here’s what I would do:
Reset it to defaults. If you access it via Winbox, you should also check the box that says no default config so it’s completely blank.
Enter this super-simple switch-only config and run your host-to-host tests. In this setup it will pass all traffic, tagged or not. Add management IP to ether1 if you like (see next config section if needed).
That’s all that should be required to make this do what you want.
This same set of configs should work on both 6.48 and later as well as 7. I recommend 7.4.1 for the CRS300 switches (as opposed to 7.5 or 7.6) due to personal experience.
CRS somehow selected “Mgmt-Bridge” to offload. Set ether1 (as bridge port) with hw=no to make “Mgmt-Bridge” non-HWoffloaded. Perhaps you’ll have to reboot switch.
After that you should see a ‘H’ in “External-Bridge” instead of current row.
Or better yet: ditch Mgmt-Bridge altogether (move IP config from bridge to stand-alone ether1 interface after dusbanding bridge).
The problem being: ROS can only offload single bridge. If there’s more than one, ROS selects bridge to offload automatically. It might happen that selection is unfortunate - as it is in your case.
And there’s no sense in having Mgmt-Bridge with single bridge port anyway.
It would have been helpful if you had posted the complete configuration with sensitive data redacted at the start.
The issue is you have two bridges and CRS3xx only supports hardware offloading on one bridge. Ideally remove Mgmt-Bridge and configure an IP address directly on ether1 for local management, or set hw=no on all bridge ports which are not members of External-Bridge
I’d like to ditch the mgmt bridge, but as this machine has no serial console, I don’t want to block access to myself, so it’s there as a reminder to not do that in future.
As I wrote: configure mgmt IP on ether1 directly. If you don’t have any restrictions regarding MAC access, then you’ll be able to connect via ether1 … you really don’t need bridge for that.
And I’ll mirror tdw’s sentiments: it would shorten the pain if you’d post full config immediately. Things are always interdependant and for inexperienced it’s impossible to tell what’s important and what not.
Use winbox and click MAC address … this way entire IP stack is bypassed for current connection. You should keep management access while changing IP settings. When you remove ether1 from bridge, you may get disconnected but you should be able to reconnect. You might need to re-run discovery in case MAC address changes (it might stay the same, depends on how exactly the bridge is defined … huh, we still need to guess and assume … see?).
And no, when tearing down mgmt-bridge, safe mode is not your friend, loosing access is expected and safe mode will then revert the changes, effectively preventing you from doing the right thing.
Excellent, I shall give that a go in the new year. It’s the last working day of the year here, and I don’t want to break anything in the last 2 hours before my break!