Slow file transfer & high cpu usage on CRS312

Below is how I configured my network on my CRS312 running on latest firmware 6.48.3

Interface 1 - WAN —> Connects from modem to the router
Interface 2 - LAN 1 that has a Bridge 1 and gets DHCP from DHCP-pool-1 range — 192.168.1.1 - 192.168.1.254
Interfaces 3 & Interface 4 - LAN 2 that has a Bridge 2 and gets DHCP from DHCP-pool-2 range — 192.168.2.1 - 192.168.2.254
Interfaces 5,6,7,8 & Combo ports viz Interfaces 9,10,11,12 - LAN 3 that has a Bridge 3 and gets DHCP from DHCP-pool-3 range — 192.168.3.1 - 192.168.3.254

I am transferring a 15gig file within 192.168.3.x subnet. I am getting a max transfer of 50 Mb/s no matter how I change the configuration. I was expecting >100mb/sec atleast that I got with my previous Ubiquiti router. I have added fast track f/w filter & fastpath is disabled because I have another firewall filter rule as below. As soon as the file transfer starts ( inspected and its smb), the cpu usage goes up to 90%. I disabled & re-enabled both rstp and hardware offloading but nothing makes any difference. I have even changed the cables but no difference. I dont know what is causing this slow transfer as my primary motive to purchase this router was to increase the transfer rates. Am I missing some fw rules or any suggestions would be really helpful.. Thank You

PC1 - 192.168.3.5 → Link rate 1Gbps - 1500 MTU - Copy to this PC from PC3 or PC2 gives me only 25Mbps
PC2 - 192.168.3.6 → Link rate 1Gbps - 1500 MTU - Copy to this PC from PC3 gives me only 50Mbps
PC3 - 192.168.3.60 → Link rate 10 Gbps - 1500 MTU - Copy to this PC from PC2 gives me only 50Mbps

/ip firewall filter print
Flags: X - disabled, I - invalid, D - dynamic
0 D ;;; special dummy rule to show fasttrack counters
chain=forward action=passthrough
1 chain=forward action=fasttrack-connection connection-state=established,related,new,untracked l
log-prefix=“”
2 chain=forward action=accept protocol=udp dst-address=192.168.3.190 in-interface=bridge2 dst-po
log-prefix=“”

/ip firewall nat print
Flags: X - disabled, I - invalid, D - dynamic
0 chain=srcnat action=masquerade out-interface=ether1 log=no log-prefix=“”
1 chain=dstnat action=dst-nat to-addresses=192.168.3.190 to-ports=9 protocol=udp in-interface=bridge2 dst-port=9 log=no log-prefix=“”

Since you mention the transfer takes place within 192.168.3.0/24, it should be pure switching, so IP firewall rules and enabled/disabled fasttracking in particular are unrelated. However, setting hw=yes on an /interface bridge port row just permits the hardware forwading, it doesn’t force it. So since you have multiple separate bridges, it is possible that RouterOS has chosen to enable hardware acceleration on other bridge than you expect. What does /interface bridge port print show?

Below is what /interface bridge port print shows. I can see HW is enabled on all the ports.

INTERFACE BRIDGE HW PVID PRIORITY PATH-COST INTERNAL-PATH-COST HORIZON

4 ether6 bridge3 yes 1 0x80 10 10 none #PC2
9 combo3 bridge3 yes 1 0x80 10 10 none #PC3

That’s really weird. Post the export according to the hint in my automatic signature below.

Here you go...

/export hide-sensitive

aug/22/2021 10:22:19 by RouterOS 6.48.3

software id = 2CMC-9V33

model = CRS312-4C+8XG

serial number = XXXXXXX

/interface bridge
add name=bridge1 protocol-mode=none
add name=bridge2 protocol-mode=none
add name=bridge3 protocol-mode=none
/interface list
add name=WAN
add name=LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip pool
add name=dhcp_pool2 ranges=192.168.2.2-192.168.2.254
add name=dhcp_pool1 ranges=192.168.1.2-192.168.1.254
add name=dhcp_pool3 ranges=192.168.3.2-192.168.3.254
/ip dhcp-server
add add-arp=yes address-pool=dhcp_pool1 disabled=no interface=bridge1 lease-time=5m name=dhcp1
add add-arp=yes address-pool=dhcp_pool2 disabled=no interface=bridge2 lease-time=5m name=dhcp2
add add-arp=yes address-pool=dhcp_pool3 disabled=no interface=bridge3 lease-time=5m name=dhcp3
/interface bridge port
add bridge=bridge1 interface=ether2
add bridge=bridge2 interface=ether3
add bridge=bridge2 interface=ether4
add bridge=bridge3 interface=ether5
add bridge=bridge3 interface=ether6
add bridge=bridge3 hw=no interface=ether7 trusted=yes
add bridge=bridge3 interface=ether8
add bridge=bridge3 interface=combo1
add bridge=bridge3 interface=combo2
add bridge=bridge3 interface=combo3
add bridge=bridge3 interface=combo4
/interface list member
add interface=ether1 list=WAN
add interface=bridge1 list=LAN
add interface=bridge2 list=LAN
add interface=bridge3 list=LAN
/ip address
add address=192.168.1.1/24 interface=bridge1 network=192.168.1.0
add address=192.168.2.1/24 interface=bridge2 network=192.168.2.0
add address=192.168.3.1/24 interface=bridge3 network=192.168.3.0
/ip arp
add address=192.168.3.190 interface=bridge3 mac-address=FF:FF:FF:FF:FF:FF
add
/ip dhcp-client
add comment=defconf disabled=no interface=ether1
/ip dhcp-server lease
add address=192.168.3.5 client-id=1:0:90:a9:ec:a:74 mac-address=00:90:A9:EC:0A:74 server=dhcp3
add address=192.168.3.6 mac-address=6C:4B:90:CD:9C:23 server=dhcp3
add address=192.168.3.25 mac-address=78:F2:9E:90:0A:11 server=dhcp3
/ip dhcp-server network
add address=192.168.1.0/24 dns-server=8.8.8.8,8.8.4.4 gateway=192.168.1.1 netmask=24
add address=192.168.2.0/24 dns-server=8.8.8.8,8.8.4.4 gateway=192.168.2.1
add address=192.168.3.0/24 dns-server=8.8.8.8,8.8.4.4 gateway=192.168.3.1
/ip firewall filter
add action=fasttrack-connection chain=forward connection-state=established,related,new,untracked
add action=accept chain=forward dst-address=192.168.3.190 dst-port=9 in-interface=bridge2 protocol=udp
/ip firewall nat
add action=masquerade chain=srcnat out-interface=ether1
add action=dst-nat chain=dstnat dst-port=9 in-interface=bridge2 protocol=udp to-addresses=192.168.3.190 to-ports=9
/ip service
set telnet address=192.168.2.0/24,192.168.3.0/24 port=2301
set ftp address=192.168.2.0/24,192.168.3.0/24 port=2101
set www disabled=yes
set ssh address=192.168.2.0/24,192.168.3.0/24 port=2201
set www-ssl disabled=no
set api disabled=yes
set winbox address=192.168.2.0/24,192.168.3.0/24,192.168.1.0/24
set api-ssl disabled=yes
/system clock
set time-zone-name=America/New_York
/system identity
set name=Mikrotik_CRS312
/system routerboard settings
set boot-os=router-os
/tool sniffer
set filter-interface=bridge2

Ah, sorry, I’ve missed that the “HW” column in the /interface bridge port print only indicates the permission of hardware acceleration, not the actual use of it. The H letter between the row index and the interface name tells you whether it is really active on that port or not.

So use /interface bridge port set [find where !(bridge=bridge3)] hw=no to disable the hardware acceleration on bridge1 and bridge2, and you should see a change:
4 H ether6 bridge3 yes 1 0x80 10 10 none #PC2
9 H combo3 bridge3 yes 1 0x80 10 10 none #PC3

Thanks a lot Sindy..That did the magic. Now I am seeing >105Mb/s between PC2 <–>PC3 & >60Mb/sec between PC1<–>PC2. CPU usage gone down to 5-10% during the transfer.

What should I do if I want to get the same transfer rates between subnet 192.168.2.x to 192.168.3.x ? I tried setting hw for bridge2 to test. Once I set, transfer rate went down in subnet 3.x and cpu usage increased.

Whereas the switching is a hardware task, routing is done by CPU, and the CPU in this model is too weak so the total routing throughput will be below 500 Mbps. So whilst you can get wirespeed switching within each subnet if you use a single bridge with multiple VLANs rather than separate bridges (only one bridge can currently benefit from hardware acceleration). To get wirespeed routing, which the switch chips used in CRS3xx models are capable of themselves, you have to give a try to RouterOS v7, which supports hardware-accelerated routing on these devices, but it is currently only available as a beta version. And I am not sure whether it will work between VLANs. As I’ve got no CRS3xx, I could not test that myself so far.

Thanks for the explanation. I will give a try with the RouterOSv7.

1.x subnet takes the web traffic, 2.x is my wireless n/w and 3.x is local lan where I have all my servers. I had the single bridge configuration and its getting very hard to maintain the rules for controlling the access rules across the servers and I dont want the servers taking the web traffic to be in the same bridge where all my files are. Hence the need for different bridges.

The IP firewall rules controlling the flow between subnets are the same regardless whether each subnet lives on a separate bridge or whether each subnet lives in a VLAN on a common bridge, you just replace the interface names in the rules (if you use them) from bridgeX to vlanX. If you make all the ports just access ones and permit only untagged and priority tagged frames on ingress, the isolation of the port groups with tagged VLANs will be the same like with port-based VLANs (aka separate bridges).

Plus you’ve currently got just a single port (ether2) in bridge1, so you can as well attach the IP configuration directly to that port rather than making it a member port of the bridge.