Community discussions

MikroTik App
 
nickscott18
just joined
Topic Author
Posts: 6
Joined: Sat Sep 18, 2021 10:15 am

Poor inter-vlan routing and High "Networking" CPU usage on RB5009

Sat Sep 18, 2021 10:42 am

Hi,

I've just got a RB5009, and am having issues with intervlan routing. My default is my access vlan, and my server, and NAS is on VLAN 40. All connection to the internal network is over the SFP+ port, and ether 1 for WAN.

When my desktop is on the sever vlan, I can get more or less wirespeed iperf tests, and ~400MB/s from my NAS. However, when my desktop is on the Access Vlan (same port on the switch, different PVID), performance drops to ~200MB/s, and IPerf tests to about 2Gb/s.

Looking at the profile when these tests are running, the "Networking" section is around 60% CPU - and minimal usage on the firewall. Is anyone able to give me a steer on where to look for this issue, my sanitized config is below, along with a diagram of the key bits of my network layout. As the tests when it's on the same vlan are traversing right back to the CRS309, I'm steering away from any hardware issues in that part of the network. As it's in the networking area, I'm wondering if it's around my bridge configuration, rather than my firewall configuration. Any help would be greatly appreciated.

# sep/18/2021 19:16:43 by RouterOS 7.1rc3
# software id = KKAD-4BXT
#
# model = RB5009UG+S+
/interface bridge
add admin-mac=2C:C8:1B:DB:5A:B4 auto-mac=no comment=defconf name=bridge pvid=30 vlan-filtering=yes
/interface pppoe-client
add add-default-route=yes disabled=no interface=ether1 name=pppoe-wan user=xxxxxxx
/interface vlan
add interface=bridge name=cameras vlan-id=60
add interface=bridge name=guest vlan-id=50
add interface=bridge name=iot vlan-id=70
add interface=bridge name=servers vlan-id=40
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
add name=RESTRICTED
/ip ipsec profile
add dpd-interval=1m enc-algorithm=aes-128 name=profile01 nat-traversal=no
/ip ipsec peer
add address=xxxxxx local-address=xxxxxx name=USG-01 profile=profile01
/ip ipsec proposal
set [ find default=yes ] enc-algorithms=aes-256-cbc,aes-128-cbc,aes-128-ctr lifetime=0s
/ip pool
add name=default-dhcp ranges=192.168.88.10-192.168.88.254
add name=mgmt-dhcp ranges=192.168.100.100-192.168.100.200
add name=access ranges=192.168.250.50-192.168.250.250
add name=servers ranges=192.168.251.100-192.168.251.200
add name=guest ranges=192.168.252.100-192.168.252.200
add name=cameras ranges=192.168.253.50-192.168.253.250
add name=iot ranges=192.168.254.100-192.168.254.200
/ip dhcp-server
add address-pool=default-dhcp disabled=yes interface=bridge name=defconf
add address-pool=mgmt-dhcp interface=ether8 name=mgmt
add address-pool=access interface=bridge lease-time=6h name=access
add address-pool=servers interface=servers lease-time=6h name=servers
add address-pool=cameras interface=cameras lease-time=6h name=cameras
add address-pool=iot interface=iot lease-time=6h name=iot
/interface bridge port
add bridge=bridge comment=defconf ingress-filtering=no interface=ether2 pvid=30
add bridge=bridge comment=defconf ingress-filtering=no interface=ether3 pvid=30
add bridge=bridge comment=defconf ingress-filtering=no interface=ether4 pvid=30
add bridge=bridge comment=defconf interface=ether5 pvid=30
add bridge=bridge comment=defconf ingress-filtering=no interface=ether6 pvid=30
add bridge=bridge comment=defconf ingress-filtering=no interface=ether7 pvid=30
add bridge=bridge comment=defconf interface=sfp-sfpplus1 pvid=30
/interface bridge settings
set use-ip-firewall-for-vlan=yes
/ip neighbor discovery-settings
set discover-interface-list=LAN
/interface bridge vlan
add bridge=bridge untagged=ether2,ether3,ether4,ether5,ether6,ether7,sfp-sfpplus1,bridge vlan-ids=30
add bridge=bridge tagged=bridge,ether2,ether3,ether4,ether5,ether6,ether7,sfp-sfpplus1 vlan-ids=40
add bridge=bridge tagged=bridge,ether2,ether3,ether4,ether5,ether6,ether7,sfp-sfpplus1 vlan-ids=50
add bridge=bridge tagged=bridge,ether2,ether3,ether4,ether5,ether6,ether7,sfp-sfpplus1 vlan-ids=60
add bridge=bridge tagged=bridge,ether2,ether3,ether4,ether5,ether6,ether7,sfp-sfpplus1 vlan-ids=70
/interface list member
add comment=defconf interface=bridge list=LAN
add comment=defconf interface=pppoe-wan list=WAN
add interface=ether8 list=LAN
add interface=guest list=LAN
add interface=servers list=LAN
add interface=iot list=RESTRICTED
add interface=cameras list=RESTRICTED
add interface=iot list=LAN
add interface=cameras list=LAN
add list=LAN
/ip address
add address=192.168.100.1/24 interface=ether8 network=192.168.100.0
add address=192.168.250.1/24 interface=bridge network=192.168.250.0
add address=192.168.251.1/24 interface=servers network=192.168.251.0
add address=192.168.252.1/24 interface=guest network=192.168.252.0
add address=192.168.253.1/24 interface=cameras network=192.168.253.0
add address=192.168.254.1/24 interface=iot network=192.168.254.0
/ip dhcp-client
add comment=defconf disabled=yes interface=ether1
/ip dhcp-server network
add address=192.168.88.0/24 comment=defconf dns-server=192.168.88.1 gateway=192.168.88.1
add address=192.168.100.0/24 dns-server=192.168.100.1 gateway=192.168.100.1
add address=192.168.250.0/24 dns-server=192.168.250.1 gateway=192.168.250.1
add address=192.168.251.0/24 dns-server=192.168.251.1 gateway=192.168.251.1
add address=192.168.252.0/24 dns-server=192.168.252.1 gateway=192.168.252.1
add address=192.168.253.0/24 dns-server=192.168.253.1 gateway=192.168.253.1
add address=192.168.254.0/24 dns-server=192.168.254.1 gateway=192.168.254.1
/ip dns
set allow-remote-requests=yes servers=8.8.8.8
/ip firewall filter
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" connection-state=established,related \
    hw-offload=yes
add action=accept chain=forward comment="defconf: accept established,related, untracked" connection-state=\
    established,related,untracked
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=accept chain=input dst-address=192.168.240.0/24 src-address=192.168.250.0/24
add action=accept chain=input src-address=xxxxxx
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment="defconf: accept to local loopback (for CAPsMAN)" dst-address=127.0.0.1
add action=drop chain=input comment="defconf: drop all not coming from LAN" in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" ipsec-policy=in,ipsec
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=forward comment="defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat \
    connection-state=new in-interface-list=WAN
add action=accept chain=forward comment="defconf: accept out ipsec policy" ipsec-policy=out,ipsec
/ip firewall nat
add action=accept chain=srcnat dst-address=192.168.240.0/24 src-address=192.168.250.0/24
add action=dst-nat chain=dstnat dst-address=xxx.xxx.xxx.xxx dst-port=443 protocol=tcp to-addresses=192.168.251.174 \
    to-ports=443
add action=dst-nat chain=dstnat dst-address=xxx.xxx.xxx.xxx dst-port=80 protocol=tcp to-addresses=192.168.251.174 \
    to-ports=80
add action=dst-nat chain=dstnat dst-address=xxx.xxx.xxx.xxx dst-port=7000 protocol=tcp to-addresses=192.168.251.157 \
    to-ports=7000
add action=dst-nat chain=dstnat dst-address=xxx.xxx.xxx.xxx dst-port=5500 protocol=tcp to-addresses=192.168.251.153 \
    to-ports=80
add action=dst-nat chain=dstnat dst-address=xxx.xxx.xxx.xxx dst-port=10022 protocol=tcp to-addresses=192.168.251.8 \
    to-ports=22
add action=dst-nat chain=dstnat dst-address=xxx.xxx.xxx.xxx dst-port=444 protocol=tcp to-addresses=192.168.251.157 \
    to-ports=443
add action=masquerade chain=srcnat dst-address=192.168.251.174 out-interface=servers protocol=tcp src-address=\
    192.168.250.0/23
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface=pppoe-wan
/ip ipsec identity
add peer=USG-01
/ip ipsec policy
set 0 disabled=yes
add dst-address=192.168.242.0/24 level=unique peer=USG-01 src-address=192.168.250.0/23 tunnel=yes
add dst-address=192.168.240.0/24 peer=USG-01 src-address=192.168.250.0/23 tunnel=yes
add dst-address=192.168.241.0/24 peer=USG-01 src-address=192.168.250.0/23 tunnel=yes
/ip traffic-flow
set active-flow-timeout=1m
/ipv6 firewall address-list
add address=::/128 comment="defconf: unspecified address" list=bad_ipv6
add address=::1/128 comment="defconf: lo" list=bad_ipv6
add address=fec0::/10 comment="defconf: site-local" list=bad_ipv6
add address=::ffff:0.0.0.0/96 comment="defconf: ipv4-mapped" list=bad_ipv6
add address=::/96 comment="defconf: ipv4 compat" list=bad_ipv6
add address=100::/64 comment="defconf: discard only " list=bad_ipv6
add address=2001:db8::/32 comment="defconf: documentation" list=bad_ipv6
add address=2001:10::/28 comment="defconf: ORCHID" list=bad_ipv6
add address=3ffe::/16 comment="defconf: 6bone" list=bad_ipv6
/ipv6 firewall filter
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMPv6" protocol=icmpv6
add action=accept chain=input comment="defconf: accept UDP traceroute" port=33434-33534 protocol=udp
add action=accept chain=input comment="defconf: accept DHCPv6-Client prefix delegation." dst-port=546 protocol=udp \
    src-address=fe80::/10
add action=accept chain=input comment="defconf: accept IKE" dst-port=500,4500 protocol=udp
add action=accept chain=input comment="defconf: accept ipsec AH" protocol=ipsec-ah
add action=accept chain=input comment="defconf: accept ipsec ESP" protocol=ipsec-esp
add action=accept chain=input comment="defconf: accept all that matches ipsec policy" ipsec-policy=in,ipsec
add action=drop chain=input comment="defconf: drop everything else not coming from LAN" in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=forward comment="defconf: drop packets with bad src ipv6" src-address-list=bad_ipv6
add action=drop chain=forward comment="defconf: drop packets with bad dst ipv6" dst-address-list=bad_ipv6
add action=drop chain=forward comment="defconf: rfc4890 drop hop-limit=1" hop-limit=equal:1 protocol=icmpv6
add action=accept chain=forward comment="defconf: accept ICMPv6" protocol=icmpv6
add action=accept chain=forward comment="defconf: accept HIP" protocol=139
add action=accept chain=forward comment="defconf: accept IKE" dst-port=500,4500 protocol=udp
add action=accept chain=forward comment="defconf: accept ipsec AH" protocol=ipsec-ah
add action=accept chain=forward comment="defconf: accept ipsec ESP" protocol=ipsec-esp
add action=accept chain=forward comment="defconf: accept all that matches ipsec policy" ipsec-policy=in,ipsec
add action=drop chain=forward comment="defconf: drop everything else not coming from LAN" in-interface-list=!LAN
/snmp
set enabled=yes
/system clock
set time-zone-name=Pacific/Auckland
/system package update
set channel=development
/system routerboard settings
set cpu-frequency=auto
/tool mac-server
set allowed-interface-list=LAN
/tool mac-server mac-winbox
set allowed-interface-list=LAN
Nick
You do not have the required permissions to view the files attached to this post.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18959
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: Poor inter-vlan routing and High "Networking" CPU usage on RB5009

Sat Sep 18, 2021 7:47 pm

Assuming your server has multiple NICs?
I would keep the default pvid of 1 and create a proper vlan30
You have 5/6 ip pools but only 4 vlans? How do you partition both management and access IP structure??

Ah okay I see, your using ether8 for management and using the bridge for access .............yechhhh
Much easier to go full vlan and NOT use bridge for anything but bridging.
vlan for management
vlan for access

Would never use these in most instances.......
/interface bridge settings
set use-ip-firewall-for-vlan=yes

Not sure why your port forwarding rules are using dst-xxx.xxx.xxx address rules, I thought PPPOE clients were dynamically assigned??

Dont particularly like your firewall rule set. the order is chaos not organized into separate chains and furthemore the logic is lacking.
For example you have very strict input chain rules such as
add action=accept chain=input src-address=xxxxxx
Then you follow it up with a last rule.
add action=drop chain=input comment="defconf: drop all not coming from LAN" in-interface-list=!LAN
Which basically states anything from the LAN to the router is just fine!!!
If your intent was to allow external source addressed access to the router that is a whole different dangerous level of wrong.
add action=drop chain=input comment="defconf: drop all not coming from LAN" in-interface-list=!LAN

Also this rule not quite clear what its purpose is.........
add action=accept chain=input dst-address=192.168.240.0/24 src-address=192.168.250.0/24

Seems like you are trying to use AN INPUT CHAIN rule to allow an existing LAN subnet access to the routers ipsec?????
Very strange to me!!
Typically one has an ipsec allow rule on the input chain to capture the initial connection from an external request PERIOD.
One can add sometimes the ipsec IP address access to the router, so one can manage/config the router, which has validity but is not the case here.
Additionally one can have a forward rule where ipsec users are given access to
a. either other LAN resources,
b. the WAN for internet access.

Thus the rule in place does not fit any category that I am aware of , but my experience is limited.

Lastly you have three sourcenat rules in this order, the only one I understand is the standard last one, not sure if order matters ........
The first one completely baffles me Incoming ipsec address is where some LAN subnet should go??, the second may have some use but looks more like a forward chain rule in the wrong place.

/ip firewall nat
add action=accept chain=srcnat dst-address=192.168.240.0/24 src-address=192.168.250.0/24
add action=masquerade chain=srcnat dst-address=192.168.251.174 out-interface=servers protocol=tcp src-address=\
192.168.250.0/23
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface=pppoe-wan
Last edited by anav on Sun Sep 19, 2021 4:01 pm, edited 4 times in total.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Poor inter-vlan routing and High "Networking" CPU usage on RB5009

Sat Sep 18, 2021 8:03 pm

Out of all @anav's suggestions, the most interesting point is "what do you expect from the use-ip-firewall-for-vlan=yes?"
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18959
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: Poor inter-vlan routing and High "Networking" CPU usage on RB5009

Sat Sep 18, 2021 8:08 pm

IM almost done sindy, you will have wait another 5minutes and the perhaps you have multiple points to bring up LOL
Okay done now!
 
nickscott18
just joined
Topic Author
Posts: 6
Joined: Sat Sep 18, 2021 10:15 am

Re: Poor inter-vlan routing and High "Networking" CPU usage on RB5009

Sun Sep 19, 2021 5:03 am

Thanks heaps for the detailed reply - I've made a change to the vlan configuration, so it's all trunked coming off the router, and the bridge is just being a bridge. Re-tested, and no noticable improvement (but, no degradation either, and a cleaner configuration, so still a win).
Assuming your server has multiple NICs?
I would keep the default pvid of 1 and create a proper vlan30
You have 5/6 ip pools but only 4 vlans? How do you partition both management and access IP structure??

Ah okay I see, your using ether8 for management and using the bridge for access .............yechhhh
Much easier to go full vlan and NOT use bridge for anything but bridging.
vlan for management
vlan for access
The "Management" interface was a poorly named interface - more like a safe access interface so I didn't lock myself out during setup. However - point taken about the mix of tagged and untagged - so, have set it up to all be full vlan (and at the same time setup a proper management vlan).
The bridge is now just a bridge (only 1 port in use however - at which stage I'm wondering if it's even worth having the bridge).
All connections to the switches are now tagged, with only the access ports being untagged.
Would never use these in most instances.......
/interface bridge settings
set use-ip-firewall-for-vlan=yes
Given "use-ip-firewall" was set to false, I assumed that would have no impact, was enabled when trying a few bits out - have disabled now to keep the config clean.

Not sure why your port forwarding rules are using dst-xxx.xxx.xxx address rules, I thought PPPOE clients were dynamically assigned??

Dont particularly like your firewall rule set. the order is chaos not organized into separate chains and furthemore the logic is lacking.
For example you have very strict input chain rules such as
add action=accept chain=input src-address=xxxxxx
Then you follow it up with a last rule.
add action=drop chain=input comment="defconf: drop all not coming from LAN" in-interface-list=!LAN
Which basically states anything from the LAN to the router is just fine!!!
If your intent was to allow external source addressed access to the router that is a whole different dangerous level of wrong.
add action=drop chain=input comment="defconf: drop all not coming from LAN" in-interface-list=!LAN
The firewall rules are a mix of default configuration, and changes to get the base config working. The "port forwarding rules, with the IP address being speficied" - this was taken from the examples at https://help.mikrotik.com/docs/display/ROS/NAT along with the hairpin nat rule ("add action=masquerade chain=srcnat dst-address=192.168.251.174 out-interface=servers protocol=tcp src-address=\
192.168.250.0/23") - am I better off changing the port forwarding rules to be based on "in-interface"? However, given I've got a static IP - this will not change until I change ISP's, I would have thought this would be functionally pretty similar.
Also this rule not quite clear what its purpose is.........
add action=accept chain=input dst-address=192.168.240.0/24 src-address=192.168.250.0/24

Seems like you are trying to use AN INPUT CHAIN rule to allow an existing LAN subnet access to the routers ipsec?????
Very strange to me!!
Typically one has an ipsec allow rule on the input chain to capture the initial connection from an external request PERIOD.
One can add sometimes the ipsec IP address access to the router, so one can manage/config the router, which has validity but is not the case here.
Additionally one can have a forward rule where ipsec users are given access to
a. either other LAN resources,
b. the WAN for internet access.

Thus the rule in place fits not category that I am aware of , but my experience is limited.
This is from the documentation for IPSec, but interesting under the section "NAT and Fasttrack bypass" - that's intriguing to me. https://wiki.mikrotik.com/wiki/Manual:IP/IPsec was the page - but given your suggestions above, I need to do more rules on this.
Lastly you have three sourcenat rules in this order, the only one I understand is the standard last one, not sure if order matters ........
The first one completely baffles me Incoming ipsec address is where some LAN subnet should go??, the second may have some use but looks more like a forward chain rule in the wrong place.

/ip firewall nat
add action=accept chain=srcnat dst-address=192.168.240.0/24 src-address=192.168.250.0/24
add action=masquerade chain=srcnat dst-address=192.168.251.174 out-interface=servers protocol=tcp src-address=\
192.168.250.0/23
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface=pppoe-wan
The first rule is mentioned above, the second rule is hairpin NAT, so I can access a couple of web services that I've got on the server from anywhere on the network.

The two bits here that are intriguing me now are in the screenshots attached - on the bridge settings "Bridge Fast Path Active" is unchecked (but looks to be a value that ROS figures out itself, and looking at the packet counters in the firewall rule, the dummy rule to pickup fast track counters isn't increasing when a traffic test is being run. Looking at this, is this indicating that fast track isn't being used? If so, I'll continue reading in that area.
You do not have the required permissions to view the files attached to this post.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Poor inter-vlan routing and High "Networking" CPU usage on RB5009

Sun Sep 19, 2021 12:53 pm

Fast track is a combination of fast path and connection tracking, and as such is only relevant for routed traffic.

If I were in this situation (given my home uplink parameters, I'm unfortunately not - no point in buying anything nearly as powerful as RB5009), I would use a step-by-step approach - first, attach the two LAN subnets directly to Ethernet interfaces; next, insert a bridge with vlan-filtering=no into the path for one of the subnets (move the IP configuration to the bridge and make the Ethernet interface the only member port of that bridge); next, set vlan-filtering to yes but still add no actual VLANs; and finally, add an /interface vlan to the bridge, move the IP configuration from the bridge to it, and set the pvid on the /interface bridge port row to the corresponding value. After one of these steps, the throughput should drop, so you'd know which type of processing is responsible for it, and could open a support ticket with Mikrotik.
 
User avatar
mozerd
Forum Veteran
Forum Veteran
Posts: 871
Joined: Thu Oct 05, 2017 3:39 pm
Location: Canada
Contact:

Re: Poor inter-vlan routing and High "Networking" CPU usage on RB5009

Sun Sep 19, 2021 1:40 pm

I have no experience with inter-vlan routing utilizing any MikroTik Switches that utilize L3 Hardware Offloading at the switch level ... having said that -- >> I would refer you to a very good reference that may help you with your RB5009 and exploiting its switch chips:

https://help.mikrotik.com/docs/display/ ... Offloading
Last edited by mozerd on Sun Sep 19, 2021 1:49 pm, edited 1 time in total.
 
User avatar
mkx
Forum Guru
Forum Guru
Posts: 11381
Joined: Thu Mar 03, 2016 10:23 pm

Re: Poor inter-vlan routing and High "Networking" CPU usage on RB5009

Sun Sep 19, 2021 1:47 pm

RB5009 doesn't support L3 HW offloading, only CRS309 does.
 
User avatar
mozerd
Forum Veteran
Forum Veteran
Posts: 871
Joined: Thu Oct 05, 2017 3:39 pm
Location: Canada
Contact:

Re: Poor inter-vlan routing and High "Networking" CPU usage on RB5009

Sun Sep 19, 2021 2:21 pm

RB5009 doesn't support L3 HW offloading, only CRS309 does.
For the Switch Chip [88E6393X] utilized in the RB5009 the following reference states that HW Offloading is supported stating with RouterOS 7.1rc1 version
HW vlan-filtering was added in the RouterOS 7.1rc1 version. The switch does not support other ether-type 0x88a8 or 0x9100 (only 0x8100 is supported) and no tag-stacking. Using these features will disable HW offload.
https://help.mikrotik.com/docs/display/ ... Offloading
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Poor inter-vlan routing and High "Networking" CPU usage on RB5009

Sun Sep 19, 2021 2:28 pm

@mozerd, HW offload of bridging and HW offload of routing are two independent features. What you quote doesn't mention the latter one.
 
User avatar
mozerd
Forum Veteran
Forum Veteran
Posts: 871
Joined: Thu Oct 05, 2017 3:39 pm
Location: Canada
Contact:

Re: Poor inter-vlan routing and High "Networking" CPU usage on RB5009

Sun Sep 19, 2021 2:33 pm

@sindy ... Thank You for correcting my misconception ... I must say that MikroTik makes my head spin :)
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Poor inter-vlan routing and High "Networking" CPU usage on RB5009

Sun Sep 19, 2021 2:47 pm

Mikrotik tries hard to keep the price tag acceptable for sensitive markets and squeeze maximum from the hardware components chosen. Which leads to this confusion, where some models support L2 offloading only if VLAN filtering is disabled, other models support it even with VLAN filtering enabled, and yet other models support even L3 offloading. And this even changes over time if a new switch chip itself does support the feature but in a different way than all the previously used ones, so the handling for the new one needs to be implemented into RouterOS.

So if the 5009's switch chip itself does support L3 forwarding, RouterOS is likely to get that feature, but it may take some time - see how long it has taken to add the L2 forwarding together with VLAN filtering to the 4011.
 
User avatar
mkx
Forum Guru
Forum Guru
Posts: 11381
Joined: Thu Mar 03, 2016 10:23 pm

Re: Poor inter-vlan routing and High "Networking" CPU usage on RB5009

Sun Sep 19, 2021 3:27 pm

A prime example of changing landscape is VLAN support on RB4011. Traditionally the only way was the software way (bridge vlan-filtering), nothing was possible through /interface ethernet switch (unlike vast majority of switch-chip equipped MT devices).
With 7.1rc1 this happened:
* added bridge HW offload support for vlan-filtering on RTL8367 switch chip (RB4011, RB1100AHx4);

Suddenly RB4011 is (configuration-wise) in the same league as CRS3xx switches.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18959
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: Poor inter-vlan routing and High "Networking" CPU usage on RB5009

Sun Sep 19, 2021 4:16 pm

I am waiting for pcunite to produce a scintillating document on how to configure the RB4011 Not the vlan filtering way......
BUT back to the OP...........

Okay Nick,
(1) Now that I understand the intent for ether8, your original idea was better. I call this my ether8-emergaccess
Where I assign an unused etherport on the router an IP address of 192.168.xx.2 (nothing else needed, no ip pool, no dhcp server etc.......)
One does NOT put this etherrnet on any bridge and a vlan is NOT required. (my bad for steering you wrong).
Then no matter what happens during configuration I can always give my laptop an IP of 192.168.xx.3 or .5 etc and get into the router!!
Hence emergency access!!
The important point here is that that etherport must be part of any rule allowing admin access to the router itself.

Thus create interface MNGMT
add ether8-emergaccess list=MNGMT
add vlanXX list=MNGMT (where vlanXX is the vlan you as the admin will be using to manage the configuration of the router).

Where is this used? In two spots!
(i) Input chain rule.
add action=accept chain=input in-interface-list=MNGMT
This can be further refined as for example vlanxx is shared with a bunch of users on a generally trusted vlan but you want to nail it down so you make a firewall address list
add ip of admin desktop list=adminaccess
add ip of admin laptop list=adminaccess
add ip of admin ipad list=adminaccess
add ip of admin smartphone list=adminaccess
add ip of emergaccess list=adminaccess (here you could add 192.168.xx.3 or .5 etc or the subnet 192.168.xx.0/24 )

So the above rule becomes
add action=accept chain=input in-interface-list=MNGMT source-address-list=adminaccess

(2) Mac Server - WINBOX MAC SERVER - interface=MNGMT
Last edited by anav on Mon Sep 20, 2021 2:50 pm, edited 3 times in total.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18959
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: Poor inter-vlan routing and High "Networking" CPU usage on RB5009

Sun Sep 19, 2021 4:20 pm

I dont even know where you got to those Bridge Settings, I dont see that on my devices.
My advice is to keep everything default!

In any case the only thing you need to do is give the bridge a unique name if you dont like the word bridge.
Then after the bridge port and bridge vlan settings are complete and matching firewall rules in place change vlan filtering from no to yES.
The router will burp but should come back to vlan filtering selected as yes.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18959
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: Poor inter-vlan routing and High "Networking" CPU usage on RB5009

Sun Sep 19, 2021 4:26 pm

Okay so good to know that you have a static WANIP, as stated I thought PPPOE provide dynamic WANIPs, since none of the experts corrected me, I assumed I was on the right track.
Perhaps they are half in the bag (heavy bout of drinking this weekend)..

In any case, yes if static, leave the destination address approach in your dstnat rules as that is correct!!

As for hairpin NAT,
(a) are you saying you have a bunch of users ON THE SAME LAN as the server??
If they are on different subnets, you DONT NEED hairpin rules!!
(b) You want them to use the WANIP address to reach the server and not the much easier LANIP? This makes sense if they are on different subnets and you dont want to make firewall forward chain rules for access via the LANIP address of the server although this would be dirt easy..

Make interface list called server-access with following members
add vlanxx list=server-access
add vlanxy list=server-access
add vlanxz list=server-access
etc.
add action=accept chain=forward in-interface-list=server-access dst-address=ipaddressOfServer

So if you still need hairpin nat.
You only need to add one sourcnat rule like this prior to the standard sourcenat rule.
(Note: For the standard rule, with a fixed WANIP/static, the format is shown. (both work but this one apparently is better))

add action=masquerade chain=srcnat comment="hairpin" dst-address=192.168.251.0/24 src-address=192.168.251.0/24
add action=src-nat chain=srcnat ipsec-policy=out,none \
out-interface-list=WAN to-addresses=fixedWANIPaddress
Last edited by anav on Mon Sep 20, 2021 2:51 pm, edited 1 time in total.
 
nickscott18
just joined
Topic Author
Posts: 6
Joined: Sat Sep 18, 2021 10:15 am

Re: Poor inter-vlan routing and High "Networking" CPU usage on RB5009

Mon Sep 20, 2021 9:41 am

Fast track is a combination of fast path and connection tracking, and as such is only relevant for routed traffic.

If I were in this situation (given my home uplink parameters, I'm unfortunately not - no point in buying anything nearly as powerful as RB5009), I would use a step-by-step approach - first, attach the two LAN subnets directly to Ethernet interfaces; next, insert a bridge with vlan-filtering=no into the path for one of the subnets (move the IP configuration to the bridge and make the Ethernet interface the only member port of that bridge); next, set vlan-filtering to yes but still add no actual VLANs; and finally, add an /interface vlan to the bridge, move the IP configuration from the bridge to it, and set the pvid on the /interface bridge port row to the corresponding value. After one of these steps, the throughput should drop, so you'd know which type of processing is responsible for it, and could open a support ticket with Mikrotik.
Yep, that is going to be my next steps. Initially I'd ruled this out as an option, but had forgotten about the 2.5gb interface. So - I'll look at moving the wan interface to another port, and use that for testing. That, and observing CPU usage at the same time should give an indication as to what's causing it.
 
nickscott18
just joined
Topic Author
Posts: 6
Joined: Sat Sep 18, 2021 10:15 am

Re: Poor inter-vlan routing and High "Networking" CPU usage on RB5009

Mon Sep 20, 2021 9:44 am

Mikrotik tries hard to keep the price tag acceptable for sensitive markets and squeeze maximum from the hardware components chosen.
This more or less summarizes why I like their hardware, and why I've gone back to it after a few years of not using it at home. For places where price is an issue, it's great - it's just a bit of a learning curve at times.
 
nickscott18
just joined
Topic Author
Posts: 6
Joined: Sat Sep 18, 2021 10:15 am

Re: Poor inter-vlan routing and High "Networking" CPU usage on RB5009

Mon Sep 20, 2021 9:54 am

First, re the suggestions for better management of the firewall rules around the management interface (and emergency access) - I really like that approach - is a nice clean way of doing it.
Okay so good to know that you have a static WANIP, as stated I thought PPPOE provide dynamic WANIPs, since none of the experts corrected me, I assumed I was on the right track.
Terminology here has probably let me down. Our provider has provided us with a static IP, that is what we see on the PPPoE interface, however it gets that address dynamically (and I assume is managed like a static dhcp lease on the providers side). The important thing to me is the address on the PPPoE interface doesn't change
Perhaps they are half in the bag (heavy bout of drinking this weekend)..


In any case, yes if static, leave the destination address approach in your dstnat rules as that is correct!!

As for hairpin NAT,
(a) are you saying you have a bunch of users ON THE SAME LAN as the server??
Kind of - but not "users". I've got a couple of services on the server network, that send data to web services hosted on the server (have a node sending to an InfluxDb node, and AppDaemon talking to HomeAssistant).

Yes, these could be exposed via split horizon DNS, and I might well end up going that way to keep the firewall rules simple. Then it's just a matter of ensuring it's all secure - the rules below will take care of that.

If they are on different subnets, you DONT NEED hairpin rules!!
(b) You want them to use the WANIP address to reach the server and not the much easier LANIP? This makes sense if they are on different subnets and you dont want to make firewall forward chain rules for access via the LANIP address of the server although this would be dirt easy..

Make interface list called server-access with following members
add vlanxx list=server-access
add vlanxx list=server-access
add vlanxx list=server-access
etc.
add action=accept chain=forward in-interface-list=server-access dst-address=ipaddressOfServer

So if you still need hairpin nat.
You only need to add one sourcnat rule like this prior to the standard sourcenat rule.
(Note: For the standard rule, with a fixed WANIP/static, the format is shown. (both work but this one apparently is better))

add action=masquerade chain=srcnat comment="hairpin" dst-address=192.168.251.0/24 src-address=192.168.251.0/24
add action=src-nat chain=srcnat ipsec-policy=out,none \
out-interface-list=WAN to-addresses=fixedWANIPaddress
And again - thanks massively for your help. I'll update in a day or so once I've done some more isolating to understand what is causing the high CPU usage, and poor performance.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18959
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: Poor inter-vlan routing and High "Networking" CPU usage on RB5009

Mon Sep 20, 2021 2:53 pm

Good to hear, I made some subtle changes to my previous post emergaccess address not being the gateway of .1, changed to .2, but very minor.
Looking forward to your feedback.

So it sounds like you have some devices on the same subnet as the server that need access to the server (and for some reason using the direct LANIP is not the right approach).
In that case a DNS query may work instead of hairpin nat as a solution.
Where you drive all DNS queries from that Subnet to the server IP.

There is one way to avoid getting into DST and Source NAT rule changes for hairpin nat and that is to use DNS.
Lets say your server IP was 192.168.88.68 and your domainname for the server was www.myserver.net
Create the following rule!
/ip dns static
add address=192.168.88.68 regexp="(^|www\\.)myserver\\.net\$" ttl=5m

The precedence for using DNS within the router is as follows...........
a. static first
b. static regexp next
c. others...

This rule will capture any request for DNS when looking for that domain name and direct the query to the server IP.
However, some users on the same subnet may have DNS hard coded on their PCs...... and thus you MAY need to redirect all DNS queries to the router to handle.

add action=redirect chain=dstnat comment="Force Users to Router for DNS - TCP" \
dst-port=53 protocol=tcp src-address=192.168.88.0/24
add action=redirect chain=dstnat comment="Force Users to Router for DNS - UDP" \
dst-port=53 protocol=udp src-address=192.168.88.0/24

This should effectively ensure that regardless of PC DNS settings, all the queries from the subnet will go through the router and thus hit the static DNS rule created.
In your case you are in control of all the devices so shouldnt be an issue but lets say those devices were hard coded to 1.1.1.1 or 8.8.8.8 then you would need the above rules
but since limited to a few you could change src-address of the subnet to source-address-list=listofdevices
 
nickscott18
just joined
Topic Author
Posts: 6
Joined: Sat Sep 18, 2021 10:15 am

Re: Poor inter-vlan routing and High "Networking" CPU usage on RB5009

Sat Oct 02, 2021 5:49 am

So it sounds like you have some devices on the same subnet as the server that need access to the server (and for some reason using the direct LANIP is not the right approach).
In that case a DNS query may work instead of hairpin nat as a solution.
Where you drive all DNS queries from that Subnet to the server IP.
Will have a ponder on that one - definitely feels like it could be easier to troubleshoot than hairpin NAT rules. I'd have a look at how I can script it, to automate pulling from my external DNS (two domains, but a handful of subdomains that I use for experimentation - and I'd rather only set it up in the one place.


In regards to the speed issues - I raised a ticket - the issue was with "VLAN Filtering". As soon as I disabled this on the bridge, the traffic went via the fast track, and performance went up, and CPU usage dropped. According to the support person, it's something they'll support in the future, just no ETA. Have struggled to find anything explicitly mentioning that in the docs - but I suspect that comes down to me, rather than it not being in the docs.

Thanks for your help, and hopefully this will help someone who has a similar issue in the future.

Who is online

Users browsing this forum: rplant and 62 guests