Community discussions

MikroTik App
 
mx5gr
just joined
Topic Author
Posts: 16
Joined: Thu Jun 22, 2017 6:02 pm

CRS326-24G-2S+ : 100% CPU utilization between networks

Sat Dec 04, 2021 3:12 pm

Hello,

I have two CRS326-24G-2S+ bonded together via a couple of trunk interfaces in a typical WAN<->Internal LAN scenario (RouterOS 6.49.1).

As the two networks (LAN/WAN) need to be separate, in order to take advantage of RouterOS HW bridge capabilities, I used a single bridge and the associated VLAN filtering feature in order to separate the two networks from each other.

The only twist to the above standard scenario is that I have a mixture of clients and servers sharing the same internal LAN network and I need to limit the access of one IP address set to the other. Thus, I also enabled IP firewall on the bridge interface.

Last but not least, as I want the internal servers/LANs to use a single as a default gateway (for redundancy reasons), I defined a common VRRP IP address between the two routers. Of course, both routers have their WAN segment connected (not to each other, via a separate cable to the WAN main switch of the building).

The issue I am having is that even when I'm copying files between the same LAN segment, I see the router's CPU used at 100%. I read that in the case of enabling IP firewall within the bridge settings, then all traffic flows through the CPU as bridge fast path is not applied. I then turned off bridge IP filtering regarding that, sacrificing on security (I cannot re-segment the internal LAN, assigning different networks to servers and clients). The same however (100% CPU utilization) appears for traffic routed between the internal LAN and the WAN interfaces. The WAN line is a Gigabit fiber one and when copying a file from an internal client to a PC within the WAN DMZ the highest throughput is around 109 Mbps with the router CPU constantly within the 90-100% range.

Please note that all ports in the bridge have the H marking on, thus showing that HW support has been enabled.

Questions:

* Can I modify the router config (with the current network setup) so that I can see better performance/throughput between ports that belong to the same LAN segment?
* Similarly, how can I achieve better throughput between the internal LAN and the WAN segment? Even if H is shown in the bridge ports, all traffic seems to flow within the CPU. Is this the answer? -> viewtopic.php?f=1&t=177092#p878135 . If there is an alternative setup (apart from VLAN filtering) in order to isolate the two networks but at the same time achieve fast track performance, please share!!
* Should we have opted out for a better switch/router should we wanted a better performance?

Here is the configuration of one of the binded CRS326 units, the other has a similar configuration. I have not included the IP Firewall rules for security reasons, I can add a couple ones as an indication should it be required.

Your help will be GREATLY appreciated!

Thank you!
/interface ethernet
set [ find default-name=ether1 ] name=ether1-WANOut
set [ find default-name=ether2 ] name=ether2-WANOut
set [ find default-name=ether3 ] name=ether3-LANin
set [ find default-name=ether4 ] name=ether4-LANin
set [ find default-name=ether4 ] name=ether5-LANin
set [ find default-name=ether23 ] name=ether23-Trunk
set [ find default-name=ether24 ] name=ether24-Trunk
set [ find default-name=sfp-sfpplus1 ] disabled=yes
set [ find default-name=sfp-sfpplus2 ] disabled=yes

/interface bridge
add admin-mac=XX:XX:XX:XX:XX:XX auto-mac=no name=bridge-LANin pvid=50 vlan-filtering=yes

/interface vrrp
add interface=bridge-LANin name=vrrp-LANin vrid=50

/interface vlan
add interface=bridge-LANin name=vlan30 vlan-id=30

/interface bonding
add lacp-rate=1sec mode=802.3ad name=bonding-switch-trunk slaves=\
    ether24-Trunk,ether23-Trunk transmit-hash-policy=layer-2-and-3

/interface list
add name=WANOut
add name=LANin-Clients
add name=LANin-Srv

/interface bridge port
add bridge=bridge-LANin edge=yes frame-types=\
    admit-only-untagged-and-priority-tagged ingress-filtering=yes interface=\
    ether1-WANOut pvid=30
add bridge=bridge-LANin edge=yes frame-types=\
    admit-only-untagged-and-priority-tagged ingress-filtering=yes interface=\
    ether2-WANOut pvid=30
add bridge=bridge-LANin frame-types=\
    admit-only-untagged-and-priority-tagged ingress-filtering=yes interface=\
    ether3-LANin pvid=50
add bridge=bridge-LANin frame-types=\
    admit-only-untagged-and-priority-tagged ingress-filtering=yes interface=\
    ether4-LANin pvid=50
add bridge=bridge-LANin interface=bonding-switch-trunk pvid=99

/interface bridge settings
set use-ip-firewall=yes

/interface bridge vlan
add bridge=bridge-LANin tagged=bonding-switch-trunk untagged="ether3-LANin,ether4-LANin,ether5-LANin,bridge-LANin,*23" vlan-ids=50
add bridge=bridge-LANin tagged=bridge-LANin untagged=ether1-WANOut,ether2-WANOut,*23 vlan-ids=30

/interface list member
add interface=ether1-WANOut list=WANOut
add interface=ether2-WANOut list=WANOut
add interface=ether3-LANin list=LANin-Srv
add interface=ether4-LANin list=LANin-Srv
add interface=ether5-LANin list=LANin-Clients
add interface=vlan30 list=WANOut

/ip address
add address=192.168.50.10/24 comment="LANin Bridge" interface=bridge-LANin network=192.168.50.0
add address=192.168.30.1/24 comment="WANOut Bridge" interface=vlan30 network=192.168.30.0
add address=192.168.50.15 comment="LANin Common" interface=vrrp-LANin network=192.168.50.0

/ip firewall address-list
add address=192.168.50.70-192.168.50.80 list=LANin-Servers-List
add address=192.168.50.20-192.168.50.50 list=LANin-Clients-List

/ip firewall nat
add action=masquerade chain=srcnat "Allow All Traffic from Clients to WANOut" out-interface-list=WANOut src-address-list=LANin-Clients-List
add action=masquerade chain=srcnat "Allow All Traffic from local Servers to WANOut" out-interface-list=WANOut src-address-list=LANin-Servers-List

 
User avatar
mkx
Forum Guru
Forum Guru
Posts: 11444
Joined: Thu Mar 03, 2016 10:23 pm

Re: CRS326-24G-2S+ : 100% CPU utilization between networks

Sat Dec 04, 2021 9:22 pm

First of all, CRS devices are essentially switches with limited capacity for routing (which will change in ROSv7), so no wonder CPU hits the ceiling when device is pushed with routing duties. The HW offload (port status H) is about switching between port belonging to same L2 domain ...
When device is routing between different IP subnets, it is not necessary to enable any bridge filtering stuff (nor use ip firewall ...) to enable use of firewall rules for traffic between different IP subnets. The bridge settings regarding IP firewall affect traffic inside same IP subnet passing bridge.

You may want to look at official test results, for real-life results look at lower-right part of the table (e.g. routing, 25 filter rules, medium packet size).
 
mx5gr
just joined
Topic Author
Posts: 16
Joined: Thu Jun 22, 2017 6:02 pm

Re: CRS326-24G-2S+ : 100% CPU utilization between networks

Sun Dec 05, 2021 4:02 pm

Thanx for the quick answer @mkx!

So, in other words, if I remove VLAN filtering within the bridge for the two different subnets (LAN/WAN) but still use VLANs to distinguish the traffic per ethernet port, the IP firewall rules as defined will continue to work? Do I understand it correctly? If this is the case, will the routing performance get increased or it will remain the same as the traffic would have to flow through the CPU for IP filtering?

As for the inter-bridge IP filtering, I am afraid that it might be a requirement in this case. The servers are within the same network as the clients (192.168.50.x/24 in the config above). We want to provide limited access of clients to servers whose ethernet ports all belong to the same VLAN (untagged) and bridge. Therefore, I believe that bridge IP filtering should be enabled in this case. If I am mistaken, please let me know...
 
User avatar
mkx
Forum Guru
Forum Guru
Posts: 11444
Joined: Thu Mar 03, 2016 10:23 pm

Re: CRS326-24G-2S+ : 100% CPU utilization between networks

Sun Dec 05, 2021 4:25 pm

So, in other words, if I remove VLAN filtering within the bridge for the two different subnets (LAN/WAN) but still use VLANs to distinguish the traffic per ethernet port, the IP firewall rules as defined will continue to work? Do I understand it correctly? If this is the case, will the routing performance get increased or it will remain the same as the traffic would have to flow through the CPU for IP filtering?

Yes, inter-VLAN firewaling will still work without "use-IP-firewall*" options set. Performance difference should not be great though ... note that if you have those options set to yes, only traffic handled by CPU will be affected anyway. As long as HW offload is active (involved ports have 'H' flag displayed), intra-VLAN traffic won't ever hit CPU.

As for the inter-bridge IP filtering, I am afraid that it might be a requirement in this case. The servers are within the same network as the clients (192.168.50.x/24 in the config above). We want to provide limited access of clients to servers whose ethernet ports all belong to the same VLAN (untagged) and bridge. Therefore, I believe that bridge IP filtering should be enabled in this case. If I am mistaken, please let me know...
You might be able to move servers to different subnet and thus be able to switch off bridge filtering.


But whatever you do, CRS3xx devices are switches with pretty capable switch chips ... but with relatively weak CPU. If device needs to do anything out of switch chip (and bridge filtering is already one of those tasks) , its performance dops to the floor.
 
mx5gr
just joined
Topic Author
Posts: 16
Joined: Thu Jun 22, 2017 6:02 pm

Re: CRS326-24G-2S+ : 100% CPU utilization between networks

Tue Dec 07, 2021 10:11 am


Yes, inter-VLAN firewaling will still work without "use-IP-firewall*" options set. Performance difference should not be great though ... note that if you have those options set to yes, only traffic handled by CPU will be affected anyway. As long as HW offload is active (involved ports have 'H' flag displayed), intra-VLAN traffic won't ever hit CPU.
Thus, if I disable VLAN filtering, there will be some performance improvement (all ports show the H letter). However, will there be an issue at L2 between the two VLANs (LAN/WAN)?

If this is the case, if we allocate another subnet for the servers (separate from the clients) and switch off bridge IP filtering, will we see HW-accelerated speeds or all traffic will still pass through the CPU?
 
User avatar
mkx
Forum Guru
Forum Guru
Posts: 11444
Joined: Thu Mar 03, 2016 10:23 pm

Re: CRS326-24G-2S+ : 100% CPU utilization between networks

Tue Dec 07, 2021 6:42 pm

None of traffic passing CPU is HW offloaded. For CRS3xx devices running ROSv6 that's anything outside same subnet. And as said many times: CRS3xx are lousy routers (anything between different VLANs is routing) ... you can get a few percent improvement with smart config, not much more.

Who is online

Users browsing this forum: Ahrefs [Bot], DanMos79, Google [Bot], jvanhambelgium and 91 guests