Hw Offloading Vlan between 2 devices

Luarion · Tue Feb 04, 2025 7:22 am

Hey, kinda newbie here, but it seems that I can't find the answer by my self.
It works, but when I do some iperf3 tests between different VLANs all the traffic seems to go through the L009UiGS's CPU and sometimes the CRS326-24G-2S+'s CPU.

I got a L009UiGS, it's in the DMZ of my ISP router via ether1.
And I configured a bond between it and a CRS326-24G-2S+ with 4 ports.
Proxmox Host on SFP+ with 10G NIC.
L3 hw offload on the CRS326-24G-2S+ enabled.

Just tell me what configs you need to see please.

Iperf3 between random Linux host on random VLAN and Proxmox host:

300~400Mb/s
L009UiGS's CPU 80+%

Iperf3 between random Proxmox VM on random VLAN and Proxmox host:

300Mb/s
L009UiGS's CPU 70+%

Iperf3 between random Proxmox VM on Proxmox host's VLAN:

20Gb/s

This is my first post, should I upload my entire config file so you can look a it?
I'm sorry in advance if I didn't explain something well, just, braining isn't braining anymore rn

mkx · Tue Feb 04, 2025 11:55 am

Some basics: L2 offload works between different ports within same VLAN. Router is needed to pass between different VLANs. Only a few devices can do L3 (routing) HW ofgload and it'll work if that device is set up as router and other devices use it as their gateway.

So it won't work by simply dropping CRS into a network ... it has to be configured as default gateway for all devices in all VLANs.

BTW traffic during the last test likely doesn't even reach the cable which connects PVE host with the rest of network.

Luarion · Wed Feb 05, 2025 12:13 am

Some basics: L2 offload works between different ports within same VLAN. Router is needed to pass between different VLANs. Only a few devices can do L3 (routing) HW ofgload and it'll work if that device is set up as router and other devices use it as their gateway.

So it won't work by simply dropping CRS into a network ... it has to be configured as default gateway for all devices in all VLANs.

BTW traffic during the last test likely doesn't even reach the cable which connects PVE host with the rest of network.

Yup, totally suspected that about my latest test.
I'm really grateful for the response.

How do you think I should approach it?
Should I configure something like OSPF (hadn't look at it enough yet) to make the CRS know the routes?
I tried adding some static routes on the CRS but seems that I don't know what I'm doing or it isn't working.

Or should I create a specific bridge for the VLANs that need to exchange traffic between them on the CRS.

mkx · Wed Feb 05, 2025 10:47 am

You want L3HW offload functional on CRS, so study this help document: https://help.mikrotik.com/docs/spaces/R ... Offloading

You'll have to add IP address to every VLAN where you want CRS to route between. And set devices in those VLANs to use CRS's address as default gateway. In simple flat network (all VLANs passing through same CRS) you don't need any of routing protocols, no need to distribute routing information ... other than to your edge router (L009). If your IP network topology won't be too dynamic, you can even live with static settings on L009, simply set static routes toward all the LAN IP subnets via CRS. If your VLAN subnets use adjacent address spaces, you can even use single route with shorter subnet mask (to cover all VLAN subnets) on L009.
Your L009 will only know about CRS (it won't be member of any of VLANs where CRS will do the routing), they will interconnect using dedicated subnet and will likely be the only two devices in that subnet. Any other device in same subnet would suffer from routing triangle at some point in time and that has potential of upseting connection tracking machinery in L009 (which is necessary for firewall functions).

So what you're up to is definitely not "kindergarten rated" and you have to understand IP routing to certain degree to get things working correctly and at right speed.

Luarion · Thu Feb 06, 2025 2:53 am

You want L3HW offload functional on CRS, so study this help document: https://help.mikrotik.com/docs/spaces/R ... Offloading

You'll have to add IP address to every VLAN where you want CRS to route between. And set devices in those VLANs to use CRS's address as default gateway. In simple flat network (all VLANs passing through same CRS) you don't need any of routing protocols, no need to distribute routing information ... other than to your edge router (L009). If your IP network topology won't be too dynamic, you can even live with static settings on L009, simply set static routes toward all the LAN IP subnets via CRS. If your VLAN subnets use adjacent address spaces, you can even use single route with shorter subnet mask (to cover all VLAN subnets) on L009.
Your L009 will only know about CRS (it won't be member of any of VLANs where CRS will do the routing), they will interconnect using dedicated subnet and will likely be the only two devices in that subnet. Any other device in same subnet would suffer from routing triangle at some point in time and that has potential of upseting connection tracking machinery in L009 (which is necessary for firewall functions).

So what you're up to is definitely not "kindergarten rated" and you have to understand IP routing to certain degree to get things working correctly and at right speed.

Alr, I think I understand it, I'll try to set it up this weekend and I'll post updates.
Really appreciate your time man, fast and clear instructions

.
One last question, if I setup all like that, does the firewall rules on the L009 still apply between the CRS VLANs?

mkx · Thu Feb 06, 2025 1:35 pm

One last question, if I setup all like that, does the firewall rules on the L009 still apply between the CRS VLANs?

No, inter-VLAN traffic will bypass L009. If you want to control inter-VLAN traffic, you have to do it on CRS .. either routing rules (these are pretty coarse, but consume way less resources) or using firewall (and you'll want to establish fasttrack offloading, not sure if your CRS supports that though). Beware that devices support a limited number of routes/connections offloaded and you don't want to end in a state where limits get exceeded.

But you do have an option to route certain VLANs via L0009 .. for those VLANs CRS will only act as a switch (without corresponding vlan interface and IP address) and L009 will be member of those VLANs. Routing table on L009 will be slightly more populated and that's all. But be prepared to fare with L009's slow routing/firewalling speed, it's nowhere near wirespeed.

Luarion · Thu Feb 06, 2025 5:18 pm

One last question, if I setup all like that, does the firewall rules on the L009 still apply between the CRS VLANs?
No, inter-VLAN traffic will bypass L009. If you want to control inter-VLAN traffic, you have to do it on CRS .. either routing rules (these are pretty coarse, but consume way less resources) or using firewall (and you'll want to establish fasttrack offloading, not sure if your CRS supports that though). Beware that devices support a limited number of routes/connections offloaded and you don't want to end in a state where limits get exceeded.

But you do have an option to route certain VLANs via L0009 .. for those VLANs CRS will only act as a switch (without corresponding vlan interface and IP address) and L009 will be member of those VLANs. Routing table on L009 will be slightly more populated and that's all. But be prepared to fare with L009's slow routing/firewalling speed, it's nowhere near wirespeed.

Alright I understand it, thanks.
Should I use the L009 to forward my services to the outside?
I got ~800Mb/s with my ISP, but when I did some speedtests it's CPU was pretty high (also the CRS) and I got around ~600Mb/s download

mkx · Thu Feb 06, 2025 6:21 pm

Your L009 is still slightly faster than CRS when it comes to CPU-based routing/firewalling (according to official test results around 40%), so it still makes sense to use it as border gateway for your home network (while using CRS as core router). Keep in mind that number of L3HW offloaded connections is limited and your "slow internet connections" might eat into limits, thus forcing local (inter-VLAN) connections to pass CPU. Which would kill your inter-VLAN performance on the spot.

Also, depending on the exact style of internet access, offered by ISP, it might not be offloaded at all (e.g. PPPoE).

Luarion · Sat Feb 08, 2025 2:00 am

Hey there.
I think CRS config it's almost done but there's something wrong going on.

CRS:

Flags: D - DYNAMIC; A - ACTIVE; c - CONNECT, s - STATIC; H - HW-OFFLOADED
Columns: DST-ADDRESS, GATEWAY, DISTANCE
#      DST-ADDRESS   GATEWAY   DISTANCE
0  AsH 0.0.0.0/0     10.1.0.1         1
  DAcH 10.1.3.0/24   vlan3            0
  DAcH 10.1.10.0/24  vlan10           0
  DAcH 10.1.11.0/24  vlan11           0
  DAcH 10.1.2.0/26   vlan2            0
  DAcH 10.1.4.0/26   vlan4            0
  DAcH 10.1.5.0/26   vlan5            0
  DAcH 10.1.21.0/26  vlan21           0
  DAcH 10.1.20.0/28  vlan20           0
  DAcH 10.1.22.0/28  vlan22           0
  DAcH 10.1.23.0/28  vlan23           0
  DAcH 10.1.0.0/30   bridge1          0

L009:

Flags: D - DYNAMIC; A - ACTIVE; c - CONNECT, d - DHCP
Columns: DST-ADDRESS, GATEWAY, DISTANCE
    DST-ADDRESS      GATEWAY       DISTANCE
DAd 0.0.0.0/0        192.168.18.1         1
DAc 192.168.18.0/24  ether1               0
DAc 192.168.88.0/24  bridge               0
DAc 10.1.0.0/30      bridge               0

As I told you, I'm a newbie in all of this, but if I ping lets say 1.1.1.1 from the CRS, It won't go out without the 0.0.0.0/0 route.
But why the f... I got my laptop connected via ethernet to the CRS and I get the DHCP lease from my L009.

Flags: D - DYNAMIC
Columns: ADDRESS, MAC-ADDRESS, HOST-NAME, SERVER, STATUS, LAST-SEEN
#   ADDRESS         MAC-ADDRESS        HOST-NAME  SERVER  STATUS  LAST-SEEN
1 D 192.168.88.225  something		L-Laptop   dhcp1   bound   13m24s

My Proxmox sits in the vlan20, inter-VLAN traffic seems to work, but it doesn't go out.

[admin@MikroTik] > ip/address/print
Flags: D - DYNAMIC; S - SLAVE
Columns: ADDRESS, NETWORK, INTERFACE
#    ADDRESS            NETWORK       INTERFACE
0    192.168.88.1/24    192.168.88.0  bridge   
1 D  192.168.18.254/24  192.168.18.0  ether1   
2  S 10.1.0.1/30        10.1.0.0      bonding1

[admin@MikroTik] > ip/address/print 
Flags: S - SLAVE
Columns: ADDRESS, NETWORK, INTERFACE
 #   ADDRESS       NETWORK    INTERFACE
;;; Home
 0   10.1.2.1/26   10.1.2.0   vlan2    
;;; IoT
 1   10.1.3.1/24   10.1.3.0   vlan3    
;;; Proxmox
 2   10.1.20.1/28  10.1.20.0  vlan20   
;;; Home Multimedia
 3   10.1.4.1/26   10.1.4.0   vlan4    
;;; Home Servers
 4   10.1.5.1/26   10.1.5.0   vlan5    
;;; Home Lab
 5   10.1.10.1/24  10.1.10.0  vlan10   
;;; Testing
 6   10.1.11.1/24  10.1.11.0  vlan11   
;;; Home Lab Servers
 7   10.1.21.1/26  10.1.21.0  vlan21   
;;; Home Public Servers
 8   10.1.22.1/28  10.1.22.0  vlan22   
 ;;; Home Game Servers
 9   10.1.23.1/28  10.1.23.0  vlan23   
10 S 10.1.0.2/30   10.1.0.0   bonding1

Luarion · Sat Feb 08, 2025 3:38 am

I made it work, except that extrange behaviour from my laptop

mkx · Sat Feb 08, 2025 3:41 pm

What is setup of port to which laptop is connected? Access or trunk? If access, which VLAN? Where is DHCP server which serves that VLAN. Or do you have DHCP relay on CRS?

Any reason for two addresses on bridge on L009? I'd remove pirt, connecting CRS, from bridge and set address directly. Or run connection as tagged and have that address on a particular vlan interface.

Luarion · Sat Feb 08, 2025 6:19 pm

What is setup of port to which laptop is connected? Access or trunk? If access, which VLAN? Where is DHCP server which serves that VLAN. Or do you have DHCP relay on CRS?

Any reason for two addresses on bridge on L009? I'd remove pirt, connecting CRS, from bridge and set address directly. Or run connection as tagged and have that address on a particular vlan interface.

The laptop was untagged on ether5 with PVID 1, now I just moved it to vlan10.
The L009 was the only DHCP server but my brain don't comprehend why all the traffic from my laptop was "like" bypassing the CRS, until I didn't add the static routes to the L009 it couldn't ping let's say 10.1.20.2, was like if I was directly connected to the L009.
I'll post you the config, I think you'll see it better than with my words.
Also when I do some speedtests the L009's CPU goes pretty high, but I think there's nothing much to do there(?).

L009:

[admin@MikroTik] > ip/address/print 
Flags: D - DYNAMIC; S - SLAVE
Columns: ADDRESS, NETWORK, INTERFACE
#    ADDRESS            NETWORK       INTERFACE
0    192.168.88.1/24    192.168.88.0  bridge   
1  S 10.1.0.1/30        10.1.0.0      bonding1 
2 D  192.168.18.254/24  192.168.18.0  ether1

I think I did what you told me to do, but I'm not sure.
Bridge it's default config, the ether1 it's getting DHCP lease from my ISP router and the bonding1 goes to the CRS (CRS got the 10.1.0.2/30).

[admin@MikroTik] > ip/route/print 
Flags: D - DYNAMIC; A - ACTIVE; c - CONNECT, s - STATIC, d - DHCP
Columns: DST-ADDRESS, GATEWAY, DISTANCE
#     DST-ADDRESS      GATEWAY       DISTANCE
  DAd 0.0.0.0/0        192.168.18.1         1
0  As 10.1.10.0/24     10.1.0.2             1
  DAc 192.168.18.0/24  ether1               0
  DAc 192.168.88.0/24  bridge               0
1  As 10.1.21.0/26     10.1.0.2             1
2  As 10.1.20.0/28     10.1.0.2             1
  DAc 10.1.0.0/30      bridge               0

(I didn't add all static routes to the CRS VLANs yet)

[admin@MikroTik] > ip/dhcp-server/print
Columns: NAME, INTERFACE, ADDRESS-POOL, LEASE-TIME
# NAME   INTERFACE  ADDRESS-POOL  LEASE-TIME
0 dhcp1  bridge     dhcp_pool15   30m

[admin@MikroTik] > interface/bridge/port/print
Flags: I - INACTIVE; H - HW-OFFLOAD
Columns: INTERFACE, BRIDGE, HW, PVID, PRIORITY, PATH-COST, INTERNAL-PATH-COST, HO>
#    INTERFACE  BRIDGE  HW   PVID  PRIORITY  PATH-COST  INTERNAL-PATH-COST  HORIZ>
;;; defconf
0 IH ether2     bridge  yes     1  0x80             10                  10  none >
;;; defconf
1 IH ether3     bridge  yes     1  0x80             10                  10  none >
;;; defconf
2 IH ether4     bridge  yes     1  0x80             10                  10  none >
;;; defconf
3 IH sfp1       bridge  yes     1  0x80             10                  10  none >
4  H bonding1   bridge  yes     1  0x80             10                  10  none >

CRS:

[admin@MikroTik] > ip/address/print
Flags: S - SLAVE
Columns: ADDRESS, NETWORK, INTERFACE
 #   ADDRESS       NETWORK    INTERFACE
;;; Home
 0   10.1.2.1/26   10.1.2.0   vlan2    
;;; IoT
 1   10.1.3.1/24   10.1.3.0   vlan3    
;;; Proxmox
 2   10.1.20.1/28  10.1.20.0  vlan20   
;;; Home Multimedia
 3   10.1.4.1/26   10.1.4.0   vlan4    
;;; Home Servers
 4   10.1.5.1/26   10.1.5.0   vlan5    
;;; Home Lab
 5   10.1.10.1/24  10.1.10.0  vlan10   
;;; Testing
 6   10.1.11.1/24  10.1.11.0  vlan11   
;;; Home Lab Servers
 7   10.1.21.1/26  10.1.21.0  vlan21   
;;; Home Public Servers
 8   10.1.22.1/28  10.1.22.0  vlan22   
;;; Home Game Servers
 9   10.1.23.1/28  10.1.23.0  vlan23   
10 S 10.1.0.2/30   10.1.0.0   bonding1

[admin@MikroTik] > ip/route/print
Flags: D - DYNAMIC; A - ACTIVE; c - CONNECT, s - STATIC; H - HW-OFFLOADED
Columns: DST-ADDRESS, GATEWAY, DISTANCE
#      DST-ADDRESS   GATEWAY   DISTANCE
0  AsH 0.0.0.0/0     10.1.0.1         1
  DAcH 10.1.3.0/24   vlan3            0
  DAcH 10.1.10.0/24  vlan10           0
  DAcH 10.1.11.0/24  vlan11           0
  DAcH 10.1.2.0/26   vlan2            0
  DAcH 10.1.4.0/26   vlan4            0
  DAcH 10.1.5.0/26   vlan5            0
  DAcH 10.1.21.0/26  vlan21           0
  DAcH 10.1.20.0/28  vlan20           0
  DAcH 10.1.22.0/28  vlan22           0
  DAcH 10.1.23.0/28  vlan23           0
  DAcH 10.1.0.0/30   bridge1          0

[admin@MikroTik] > interface/bridge/port/print
Flags: I - INACTIVE; D - DYNAMIC; H - HW-OFFLOAD
Columns: INTERFACE, BRIDGE, HW, PVID, PRIORITY, HORIZON
 #     INTERFACE     BRIDGE   HW   PVID  PRIORITY  HORIZON
 0   H bonding1      bridge1  yes     1  0x80      none   
 1     Home          bridge1  yes     2  0x80      none   
 2 IDH ether1        bridge1  yes     2  0x80      none   
 3 IDH ether2        bridge1  yes     2  0x80      none   
 4 IDH ether3        bridge1  yes     2  0x80      none   
 5 IDH ether4        bridge1  yes     2  0x80      none   
 6 IDH ether6        bridge1  yes     2  0x80      none   
 7 IDH ether7        bridge1  yes     2  0x80      none   
 8     Proxmox       bridge1  yes    20  0x80      none   
 9  DH sfp-sfpplus2  bridge1  yes    20  0x80      none   
10     Home Lab      bridge1  yes    10  0x80      none   
11  DH ether5        bridge1  yes    10  0x80      none

mkx · Sat Feb 08, 2025 6:34 pm

Post output of /export command (redact sensitive information, such as serial number) ... print's show running config but not how it ended up being like that.

Re. L009 CPU load: L009 has moderate routing capacity (for today's standards) of something between 300Mbps and 2Gbps depending on the actual config ... with anything in firewall it'll be in the lower part of that range. So yes, when L009 has to route traffic, its CPU will likely be the bottleneck. And the point of setting CRS to route between VLANs is to bypass slow L009 with (fast due to L3HW offload) CRS to do the inter-VLAN routing.

mkx · Sat Feb 08, 2025 6:39 pm

(I didn't add all static routes to the CRS VLANs yet)

Well, you'll have to. You can't expect the "jolly new roundabout" fully functional if you're letting traffic reach it via some old goat path. And even if traffic does flow somehow (partly via new roundabout, partly old goat path), you can't assess how well the new path works. So until you finish rebuilding your paths/routing, your only concern should be to retain management access to your devices.

Luarion · Sat Feb 08, 2025 7:09 pm

Post output of /export command (redact sensitive information, such as serial number) ... print's show running config but not how it ended up being like that.

Ty for the tip

[admin@MikroTik] > export
# 2025-02-08 18:04:29 by RouterOS 7.17.2
# software id = ""
#
# model = L009UiGS
# serial number = ""
/interface bridge
add admin-mac=D4:01:C3:93:CF:97 auto-mac=no name=bridge port-cost-mode=short \
    vlan-filtering=yes
/interface bonding
add comment="To Switch" mode=802.3ad name=bonding1 slaves=\
    ether5,ether6,ether7,ether8
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip pool
add name=default-dhcp ranges=192.168.88.10-192.168.88.254
add comment=Home name=dhcp_pool2 ranges=10.1.2.2-10.1.2.30
add name=dhcp_pool20 ranges=10.1.20.2-10.1.20.6
add name=dhcp_pool10 ranges=10.1.10.2-10.1.10.254
add name=dhcp_pool13 ranges=10.1.5.2-10.1.5.14
add name=dhcp_pool14 ranges=10.0.0.1-10.1.11.0,10.1.11.2-10.255.255.254
add name=dhcp_pool15 ranges=192.168.88.2-192.168.88.254
/ip dhcp-server
add address-pool=dhcp_pool15 interface=bridge name=dhcp1
/port
set 0 name=serial0
/ppp profile
set *FFFFFFFE bridge=bridge dns-server=1.1.1.1,8.8.8.8 local-address=\
    192.168.88.1 remote-address=192.168.88.69
/interface bridge port
add bridge=bridge comment=defconf interface=ether2 internal-path-cost=10 \
    path-cost=10
add bridge=bridge comment=defconf interface=ether3 internal-path-cost=10 \
    path-cost=10
add bridge=bridge comment=defconf interface=ether4 internal-path-cost=10 \
    path-cost=10
add bridge=bridge comment=defconf interface=sfp1 internal-path-cost=10 \
    path-cost=10
add bridge=bridge interface=bonding1 internal-path-cost=10 path-cost=10
/ip firewall connection tracking
set udp-timeout=10s
/ip neighbor discovery-settings
set discover-interface-list=LAN
/interface bridge vlan
add bridge=bridge tagged=bonding1 vlan-ids=1-5,10-11,20-23
/interface l2tp-server server
set authentication=mschap2 enabled=yes use-ipsec=yes
/interface list member
add comment=defconf interface=bridge list=LAN
add comment=defconf interface=ether1 list=WAN
/interface ovpn-server server
add mac-address=[quote=mkx]
[quote=Luarion post_id=1124878 time=1739031558 user_id=246035]
(I didn't add all static routes to the CRS VLANs yet)
[/quote]


Well, you'll have to. You can't expect the "jolly new roundabout" fully functional if you're letting traffic reach it via some old goat path. And even if traffic does flow somehow (partly via new roundabout, partly old goat path), you can't assess how well the new path works. So until you finish rebuilding your paths/routing, your only concern should be to retain management access to your devices.
[/quote] name=ovpn-server1
/interface pptp-server server
# PPTP connections are considered unsafe, it is suggested to use a more modern VPN 
 rotocol instead
set enabled=yes
/interface wireguard peers
add allowed-address=10.1.255.2/24 interface=*25 name=admin public-key=\
    ""
/ip address
add address=192.168.88.1/24 interface=bridge network=192.168.88.0
add address=10.1.0.1/30 interface=bonding1 network=10.1.0.0
/ip cloud
set ddns-enabled=yes ddns-update-interval=1m
/ip dhcp-client
add comment=defconf interface=ether1
/ip dhcp-server network
add address=10.0.0.0/8 gateway=10.1.11.1
add address=10.1.2.0/27 gateway=10.1.2.1
add address=10.1.5.0/28 gateway=10.1.5.1
add address=10.1.10.0/24 gateway=10.1.10.1
add address=10.1.20.0/29 gateway=10.1.20.1
add address=10.1.23.0/29 gateway=10.1.23.1
add address=192.168.88.0/24 comment=defconf dns-server=192.168.88.1 gateway=\
    192.168.88.1
/ip dns
set allow-remote-requests=yes cache-size=20480KiB servers=\
    1.1.1.1,8.8.8.8,8.8.4.4
/ip dns adlist
add ssl-verify=no url=\
    https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
add ssl-verify=no url="https://www.github.developerdan.com/hosts/lists/ads-and-tr\
    acking-extended.txt"
/ip dns static
add address=192.168.88.1 comment=defconf name=router.lan type=A
/ip firewall address-list
add address=10.1.23.2 list="Minecraft Servers"
add address=10.1.10.0/24 list="All Lab"
add address=10.1.21.0/28 list="All Lab"
add address=10.1.23.0/28 list="Game Servers"
add address=authserver.mojang.com list="Minecraft Auth Servers"
add address=sessionserver.mojang.com list="Minecraft Auth Servers"
/ip firewall filter
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
    invalid
add action=accept chain=input comment=PPTP in-interface-list=WAN port=1723 \
    protocol=tcp
add action=accept chain=input comment=PPTP in-interface-list=WAN port=1723 \
    protocol=udp
add action=accept chain=input dst-port=8291 in-interface-list=!WAN protocol=tcp \
    src-address=192.168.88.69
add action=accept chain=input dst-address=!192.168.88.1 dst-port=8291 \
    in-interface-list=!WAN protocol=tcp src-address-list="All Lab"
add action=accept chain=input comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment=\
    "defconf: accept to local loopback (for CAPsMAN)" dst-address=127.0.0.1
add action=drop chain=input comment="defconf: drop all not coming from LAN" \
    in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" \
    ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" \
    ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" \
    connection-state=established,related hw-offload=yes
add action=accept chain=forward comment=\
    "defconf: accept established,related, untracked" connection-state=\
    established,related,untracked
add action=accept chain=input comment=WireGuard dst-port=13231 \
    in-interface-list=WAN protocol=udp
add action=drop chain=input comment=Ping disabled=yes in-interface-list=WAN \
    protocol=icmp
add action=accept chain=input comment=L2TP disabled=yes in-interface-list=WAN \
    protocol=ipsec-esp
add action=accept chain=input comment=L2TP disabled=yes in-interface-list=WAN \
    protocol=ipsec-ah
add action=accept chain=input comment=L2TP disabled=yes dst-port="" \
    in-interface-list=WAN port=500,4500,1701 protocol=udp
add action=accept chain=forward comment="Game Servers Minecraft In" disabled=yes \
    dst-address-list="Game Servers" dst-port=25565 protocol=tcp
add action=accept chain=forward comment="Game Servers Minecraft In" disabled=yes \
    dst-address-list="Game Servers" dst-port=25565 protocol=udp
add action=accept chain=forward comment="Game Servers Minecraft Out" disabled=\
    yes protocol=tcp src-address-list="Game Servers" src-port=25565
add action=accept chain=forward comment="Game Servers Minecraft Out" disabled=\
    yes protocol=udp src-address-list="Game Servers" src-port=25565
add action=accept chain=forward comment="Game Servers Minecraft Auth Servers" \
    disabled=yes dst-address-list="Minecraft Auth Servers" dst-port=443 \
    protocol=tcp src-address-list="Game Servers"
add action=accept chain=forward comment="Game Servers DNS" disabled=yes \
    dst-port=53 out-interface-list=WAN protocol=udp src-address-list=\
    "Game Servers"
add action=accept chain=forward comment="Game Servers SSH In" disabled=yes \
    dst-address-list="Game Servers" dst-port=22 protocol=tcp
add action=accept chain=forward comment="Game Servers SSH Out" disabled=yes \
    protocol=tcp src-address-list="Game Servers" src-port=22
add action=accept chain=forward comment="Game Servers Ping In" disabled=yes \
    dst-address-list="Game Servers" protocol=icmp
add action=accept chain=forward comment="Game Servers Ping Out" disabled=yes \
    protocol=icmp src-address-list="Game Servers"
add action=reject chain=forward comment="Game Servers In" disabled=yes \
    dst-address-list="Game Servers" reject-with=icmp-admin-prohibited
add action=reject chain=forward comment="Game Servers Out" disabled=yes \
    reject-with=icmp-admin-prohibited src-address-list="Game Servers"
add action=drop chain=forward comment="defconf: drop invalid" connection-state=\
    invalid
add action=drop chain=forward comment="defconf: drop all from WAN not DSTNATed" \
    connection-nat-state=!dstnat connection-state=new in-interface-list=WAN
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=\
    out,none out-interface-list=WAN
add action=masquerade chain=srcnat src-address=192.168.88.69
add action=dst-nat chain=dstnat comment=Minecraft dst-port=25565 \
    in-interface-list=WAN protocol=tcp to-addresses=10.1.23.2 to-ports=25565
add action=dst-nat chain=dstnat comment=Minecraft dst-port=25565 \
    in-interface-list=WAN protocol=udp to-addresses=10.1.23.2 to-ports=25565
add action=dst-nat chain=dstnat comment=Proxmox dst-port=8006 in-interface-list=\
    WAN protocol=tcp to-addresses=10.1.20.2 to-ports=8006
add action=dst-nat chain=dstnat comment=Proxmox dst-port=8006 in-interface-list=\
    WAN protocol=udp to-addresses=10.1.20.2 to-ports=8006
/ip hotspot profile
set [ find default=yes ] html-directory=hotspot
/ip ipsec profile
set [ find default=yes ] dpd-interval=2m dpd-maximum-failures=5
/ip route
add disabled=no distance=1 dst-address=10.1.20.0/28 gateway=10.1.0.2 \
    routing-table=main scope=30 suppress-hw-offload=no target-scope=10
add disabled=no distance=1 dst-address=10.1.21.0/26 gateway=10.1.0.2 \
    routing-table=main scope=30 suppress-hw-offload=no target-scope=10
add disabled=no distance=1 dst-address=10.1.10.0/24 gateway=10.1.0.2 \
    routing-table=main scope=30 suppress-hw-offload=no target-scope=10
/ipv6 firewall address-list
add address=::/128 comment="defconf: unspecified address" list=bad_ipv6
add address=::1/128 comment="defconf: lo" list=bad_ipv6
add address=fec0::/10 comment="defconf: site-local" list=bad_ipv6
add address=::ffff:0.0.0.0/96 comment="defconf: ipv4-mapped" list=bad_ipv6
add address=::/96 comment="defconf: ipv4 compat" list=bad_ipv6
add address=100::/64 comment="defconf: discard only " list=bad_ipv6
add address=2001:db8::/32 comment="defconf: documentation" list=bad_ipv6
add address=2001:10::/28 comment="defconf: ORCHID" list=bad_ipv6
add address=3ffe::/16 comment="defconf: 6bone" list=bad_ipv6
/ipv6 firewall filter
add action=accept chain=input comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
    invalid
add action=accept chain=input comment="defconf: accept ICMPv6" protocol=icmpv6
add action=accept chain=input comment="defconf: accept UDP traceroute" port=\
    33434-33534 protocol=udp
add action=accept chain=input comment=\
    "defconf: accept DHCPv6-Client prefix delegation." dst-port=546 protocol=udp \
    src-address=fe80::/10
add action=accept chain=input comment="defconf: accept IKE" dst-port=500,4500 \
    protocol=udp
add action=accept chain=input comment="defconf: accept ipsec AH" protocol=\
    ipsec-ah
add action=accept chain=input comment="defconf: accept ipsec ESP" protocol=\
    ipsec-esp
add action=accept chain=input comment=\
    "defconf: accept all that matches ipsec policy" ipsec-policy=in,ipsec
add action=drop chain=input comment=\
    "defconf: drop everything else not coming from LAN" in-interface-list=!LAN
add action=accept chain=forward comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=\
    invalid
add action=drop chain=forward comment="defconf: drop packets with bad src ipv6" \
    src-address-list=bad_ipv6
add action=drop chain=forward comment="defconf: drop packets with bad dst ipv6" \
    dst-address-list=bad_ipv6
add action=drop chain=forward comment="defconf: rfc4890 drop hop-limit=1" \
    hop-limit=equal:1 protocol=icmpv6
add action=accept chain=forward comment="defconf: accept ICMPv6" protocol=icmpv6
add action=accept chain=forward comment="defconf: accept HIP" protocol=139
add action=accept chain=forward comment="defconf: accept IKE" dst-port=500,4500 \
    protocol=udp
add action=accept chain=forward comment="defconf: accept ipsec AH" protocol=\
    ipsec-ah
add action=accept chain=forward comment="defconf: accept ipsec ESP" protocol=\
    ipsec-esp
add action=accept chain=forward comment=\
    "defconf: accept all that matches ipsec policy" ipsec-policy=in,ipsec
add action=drop chain=forward comment=\
    "defconf: drop everything else not coming from LAN" in-interface-list=!LAN
/ppp secret
add local-address=192.168.88.1 name=admin profile=default-encryption service=\
    pptp
/system clock
set time-zone-name=Europe/Madrid
/system note
set show-at-login=no
/system routerboard settings
set enter-setup-on=delete-key
/tool graphing interface
add interface=ether1
/tool mac-server
set allowed-interface-list=LAN
/tool mac-server mac-winbox
set allowed-interface-list=LAN
/tool romon
set enabled=yes

[admin@MikroTik] > export
# 2025-02-08 18:01:57 by RouterOS 7.17.2
# software id = ""
#
# model = CRS326-24G-2S+
# serial number = ""
/interface bridge
add name=bridge1 vlan-filtering=yes
/interface ethernet
set [ find default-name=ether1 ] comment="Sal\C3\B3n"
set [ find default-name=ether2 ] comment="Habitaci\F3n Invitados"
set [ find default-name=ether3 ] comment=Cocina
set [ find default-name=ether4 ] comment="TV Ra\C3\BAl"
set [ find default-name=ether5 ] comment="Desktop Ra\C3\BAl"
set [ find default-name=ether6 ] comment="Habitaci\C3\B3n Matrimonio"
set [ find default-name=ether7 ] comment=Despacho
set [ find default-name=sfp-sfpplus1 ] disabled=yes
/interface vlan
add comment="\F0\9F\8F\A1 Home" interface=bridge1 name=vlan2 vlan-id=2
add comment=IoT interface=bridge1 name=vlan3 vlan-id=3
add comment="\F0\9F\93\BA Home Multimedia" interface=bridge1 name=vlan4 vlan-id=\
    4
add comment="Home Servers" interface=bridge1 name=vlan5 vlan-id=5
add comment="\F0\9F\A7\AA Home Lab" interface=bridge1 name=vlan10 vlan-id=10
add comment=Testing interface=bridge1 name=vlan11 vlan-id=11
add comment=Proxmox interface=bridge1 name=vlan20 vlan-id=20
add comment="Home Lab Servers" interface=bridge1 name=vlan21 vlan-id=21
add comment="Home Public Servers" interface=bridge1 name=vlan22 vlan-id=22
add comment="Home Game Servers" interface=bridge1 name=vlan23 vlan-id=23
/interface bonding
add comment="To L009" mode=802.3ad name=bonding1 slaves=\
    ether21,ether22,ether23,ether24
/interface list
add name=Home
add name="Home Lab"
add name=Proxmox
/ip pool
add name=dhcp_pool10 ranges=10.1.10.2-10.1.10.254
/ip dhcp-server
add address-pool=dhcp_pool10 interface=vlan10 name=dhcp10
/port
set 0 name=serial0
/interface bridge port
add bridge=bridge1 interface=bonding1
add bridge=bridge1 interface=Home pvid=2
add bridge=bridge1 interface=Proxmox pvid=20
add bridge=bridge1 interface="Home Lab" pvid=10
/interface bridge vlan
add bridge=bridge1 tagged=Proxmox vlan-ids=2-5,11,21-23
add bridge=bridge1 tagged=Proxmox vlan-ids=10
/interface ethernet switch
set 0 l3-hw-offloading=yes qos-hw-offloading=yes
/interface list member
add interface=ether7 list=Home
add interface=ether6 list=Home
add interface=ether4 list=Home
add interface=ether3 list=Home
add interface=ether2 list=Home
add interface=ether1 list=Home
add interface=sfp-sfpplus2 list=Proxmox
add interface=ether5 list="Home Lab"
/ip address
add address=10.1.2.1/26 comment=Home interface=vlan2 network=10.1.2.0
add address=10.1.3.1/24 comment=IoT interface=vlan3 network=10.1.3.0
add address=10.1.20.1/28 comment=Proxmox interface=vlan20 network=10.1.20.0
add address=10.1.4.1/26 comment="Home Multimedia" interface=vlan4 network=\
    10.1.4.0
add address=10.1.5.1/26 comment="Home Servers" interface=vlan5 network=10.1.5.0
add address=10.1.10.1/24 comment="Home Lab" interface=vlan10 network=10.1.10.0
add address=10.1.11.1/24 comment=Testing interface=vlan11 network=10.1.11.0
add address=10.1.21.1/26 comment="Home Lab Servers" interface=vlan21 network=\
    10.1.21.0
add address=10.1.22.1/28 comment="Home Public Servers" interface=vlan22 network=\
    10.1.22.0
add address=10.1.23.1/28 comment="Home Game Servers" interface=vlan23 network=\
    10.1.23.0
add address=10.1.0.2/30 interface=bonding1 network=10.1.0.0
/ip dhcp-server lease
add address=10.1.10.2 client-id=1:0:1a:4d:11:d4:ec mac-address=00:1A:4D:11:D4:EC \
    server=dhcp10
/ip dhcp-server network
add address=10.1.10.0/24 gateway=10.1.10.1
/ip dns
set servers=1.1.1.1
/ip firewall filter
add action=fasttrack-connection chain=forward comment=fasttrack \
    connection-state=established,related hw-offload=yes
/ip hotspot profile
set [ find default=yes ] html-directory=hotspot
/ip route
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=10.1.0.1 routing-table=\
    main scope=30 suppress-hw-offload=no target-scope=10
/system clock
set time-zone-name=Europe/Madrid
/system note
set show-at-login=no
/system routerboard settings
set enter-setup-on=delete-key
/tool romon
set enabled=yes

mkx · Sat Feb 08, 2025 7:33 pm

Just had a look at CRS config ... and it's lacking a lot vith regards to VLAN setup. Did you ever go through this tutorial? viewtopic.php?t=143620 Read the "router" section, that's what CRS should become eventually.

The biggest issue: bridge1 CPU-facing port has to be tagged member of all VLANs where switch is supposed to route between.
Same goes for bond1 (I'm guessing thete will be some tagged VLANs passing between CRS and L009).

Luarion · Mon Feb 10, 2025 5:05 pm

Just had a look at CRS config ... and it's lacking a lot vith regards to VLAN setup. Did you ever go through this tutorial? viewtopic.php?t=143620 Read the "router" section, that's what CRS should become eventually.

Thanks for sharing that tutorial, finally got it working.
I think another issue it's the 4 port bond, seems like it's struggling, since I only need less than 1G I'll just set 2 ports as active backup and I'll update if it works better or not.

I'm also having some trouble with VPNs behind DMZ, I just created another post if you want to check: viewtopic.php?t=214625

Hw Offloading Vlan between 2 devices [SOLVED]

Hw Offloading Vlan between 2 devices

Re: Hw Offloading Vlan between 2 devices

Re: Hw Offloading Vlan between 2 devices

Re: Hw Offloading Vlan between 2 devices

Re: Hw Offloading Vlan between 2 devices

Re: Hw Offloading Vlan between 2 devices

Re: Hw Offloading Vlan between 2 devices

Re: Hw Offloading Vlan between 2 devices

Re: Hw Offloading Vlan between 2 devices

Re: Hw Offloading Vlan between 2 devices

Re: Hw Offloading Vlan between 2 devices

Re: Hw Offloading Vlan between 2 devices

Re: Hw Offloading Vlan between 2 devices

Re: Hw Offloading Vlan between 2 devices

Re: Hw Offloading Vlan between 2 devices

Re: Hw Offloading Vlan between 2 devices [SOLVED]

Re: Hw Offloading Vlan between 2 devices