Dual WAN PCC ok but no web browsing

Hello all,

I’ve got two WAN connections: a Starlink and an FWA, both working between 150 and 300Mbps each one. There are two hEX routers in VRRP, acting with PCC. The link aggregation works very well as performances (i.e. 500Mbps), but when accessing a website, ping replies ok, while the page content could take several minutes to appear; once appeared, browsing between other pages same site is okay.
Tried to delete active connections, but no changes.
By preceding the PCC mangle cascade with an access rule, so bypassing it and using the Starlink only, the connection is very responsive as it should do.
I’ve found several posts describing this behaviour, but none of them helped me in solving that.

Starlink router: 192.168.1.1
FWA router: 192.168.55.1

Here my config:

/ip firewall mangle
add action=mark-connection chain=input connection-mark=no-mark connection-state=new in-interface=vrrp2-SAT new-connection-mark=SAT1_conn passthrough=yes
add action=mark-connection chain=input connection-mark=no-mark connection-state=new in-interface=vrrp3-FWA new-connection-mark=FWA1_conn passthrough=yes
add action=mark-routing chain=output connection-mark=SAT1_conn new-routing-mark=to_SAT1 passthrough=yes
add action=mark-routing chain=output connection-mark=FWA1_conn new-routing-mark=to_FWA1 passthrough=yes
add action=mark-connection chain=prerouting connection-mark=no-mark connection-state=new dst-address-type=!local in-interface=vrrp1-LAN new-connection-mark=SAT1_conn passthrough=yes per-connection-classifier=src-address-and-port:2/0
add action=mark-connection chain=prerouting connection-mark=no-mark connection-state=new dst-address-type=!local in-interface=vrrp1-LAN new-connection-mark=FWA1_conn passthrough=yes per-connection-classifier=src-address-and-port:2/1
add action=mark-routing chain=prerouting connection-mark=SAT1_conn in-interface=vrrp1-LAN new-routing-mark=to_SAT1 passthrough=no
add action=mark-routing chain=prerouting connection-mark=FWA1_conn in-interface=vrrp1-LAN new-routing-mark=to_FWA1 passthrough=no

/ip route
add check-gateway=ping disabled=no distance=1 dst-address=0.0.0.0/0 gateway=192.168.1.1 pref-src=“” routing-table=to_SAT1 scope=30 suppress-hw-offload=no target-scope=10
add check-gateway=ping disabled=no distance=1 dst-address=0.0.0.0/0 gateway=192.168.55.1 pref-src=“” routing-table=to_FWA1 scope=30 suppress-hw-offload=no target-scope=10
add check-gateway=ping disabled=no distance=1 dst-address=0.0.0.0/0 gateway=192.168.1.1 pref-src=“” routing-table=main scope=30 suppress-hw-offload=no target-scope=10
add check-gateway=ping disabled=no distance=2 dst-address=0.0.0.0/0 gateway=192.168.55.1 pref-src=“” routing-table=main scope=30 suppress-hw-offload=no target-scope=10

/ip firewall nat
add action=masquerade chain=srcnat out-interface=vrrp2-SAT
add action=masquerade chain=srcnat out-interface=vrrp3-FWA
add action=redirect chain=dstnat dst-address-type=!local dst-port=53 protocol=udp to-ports=53
add action=redirect chain=dstnat dst-address-type=!local dst-port=53 protocol=tcp to-ports=53

Any help would be highly appreciated
Best
Luca

What about other config? Is there perhaps fasttrack in /ip firewall filter?

Fasttrack is not present. The config is slightly changed from the posted one due to some tests.

/ip firewall address-list
add address=gse.it list=direct
add address=173.249.28.133 list=direct
add address=40.113.176.167 list=direct
add address=54.78.248.218 list=direct
add address=34.252.18.142 list=direct
add address=10.0.0.0/8 list=RFC1918
add address=172.16.0.0/12 list=RFC1918
add address=192.168.0.0/16 list=RFC1918
/ip firewall connection tracking
set enabled=yes tcp-close-wait-timeout=1m tcp-established-timeout=1h tcp-fin-wait-timeout=1m tcp-last-ack-timeout=30s tcp-syn-received-timeout=1m tcp-syn-sent-timeout=10s tcp-time-wait-timeout=1m udp-timeout=30s
/ip firewall filter
add action=drop chain=input connection-state=invalid
add action=drop chain=forward comment=NetBIOS dst-port=137-139,445 in-interface=bridge1 protocol=tcp
add action=drop chain=forward dst-port=137-139,445 in-interface=bridge1 protocol=udp
add action=drop chain=forward dst-address=192.168.10.0/24 src-address=192.168.50.0/24
add action=drop chain=input dst-port=53 in-interface=vrrp2-SAT protocol=tcp
add action=drop chain=input dst-port=53 in-interface=vrrp3-FWA protocol=tcp
add action=accept chain=forward connection-state=established,related,untracked in-interface=all-ethernet
add action=accept chain=forward dst-address=192.168.10.7 dst-port=53 protocol=udp
add action=accept chain=forward dst-address=192.168.10.6 dst-port=53 protocol=udp
add action=accept chain=input in-interface=all-ethernet
add action=accept chain=output out-interface=all-ethernet
/ip firewall mangle
add action=accept chain=prerouting comment=“no PCC ports” dst-port=8883 in-interface=vrrp1-LAN protocol=tcp
add action=accept chain=prerouting comment=“no PCC destinations” dst-address-list=direct in-interface=vrrp1-LAN
add action=mark-connection chain=input comment=“Sticky connections” connection-mark=no-mark connection-state=new in-interface=vrrp2-SAT new-connection-mark=SAT1_conn passthrough=yes
add action=mark-connection chain=input connection-mark=no-mark connection-state=new in-interface=vrrp3-FWA new-connection-mark=FWA1_conn passthrough=yes
add action=mark-routing chain=output comment=“Output chain” connection-mark=SAT1_conn new-routing-mark=to_SAT1 passthrough=yes
add action=mark-routing chain=output connection-mark=FWA1_conn new-routing-mark=to_FWA1 passthrough=yes
add action=accept chain=prerouting comment=“Accept rule” dst-address-list=RFC1918 in-interface=vrrp2-SAT
add action=accept chain=prerouting dst-address-list=RFC1918 in-interface=vrrp3-FWA
add action=accept chain=prerouting dst-address-type=local src-address-type=local
add action=mark-connection chain=prerouting comment=PCC connection-mark=no-mark connection-state=new dst-address-type=!local in-interface=vrrp1-LAN new-connection-mark=SAT1_conn passthrough=no per-connection-classifier=src-address-and-port:2/0 src-address-type=!local
add action=mark-connection chain=prerouting connection-mark=no-mark connection-state=new dst-address-type=!local in-interface=vrrp1-LAN new-connection-mark=FWA1_conn passthrough=no per-connection-classifier=src-address-and-port:2/1 src-address-type=!local
add action=mark-routing chain=prerouting connection-mark=SAT1_conn in-interface=vrrp1-LAN new-routing-mark=to_SAT1 passthrough=yes
add action=mark-routing chain=prerouting connection-mark=FWA1_conn in-interface=vrrp1-LAN new-routing-mark=to_FWA1 passthrough=yes
/ip firewall nat
add action=masquerade chain=srcnat comment=PCC out-interface=vrrp2-SAT to-addresses=192.168.1.1
add action=masquerade chain=srcnat out-interface=vrrp3-FWA to-addresses=192.168.55.1
add action=redirect chain=dstnat comment=DNS dst-address-type=!local dst-port=53 protocol=udp to-ports=53
add action=redirect chain=dstnat dst-address-type=!local dst-port=53 protocol=tcp to-ports=53

Question: is PCC supported with VRRP?

Thank you!
Luca

PCC with VRRP, I don’t see why not. But it’s probably good idea to share more info about what exactly you have (at least definition of interfaces and IP addresses; or just post whole config). You seem to have VRRPs on LAN and both WANs. But one weird thing I see, if srcnat is intended to be there and it’s not just some leftover, normally the outgoing interface would be the parent one, not VRRP. So something is not as I’d expect.

And your /ip firewall filter is half-useless mess, but it’s not breaking it.

Ok, my thought was to make the config as simple as possible to read. Here below the full config.
About the srcnat, already tried setting the parent interface, without success.

Addresses:

LAN 192.168.10.0/24
SAT1 192.168.1.1/24
FWA1 192.168.55.1/24

/interface bridge
add ingress-filtering=no name=bridge1 port-cost-mode=short vlan-filtering=yes
/interface ethernet
set [ find default-name=ether1 ] name=ether1-bbone rx-flow-control=auto
tx-flow-control=auto
set [ find default-name=ether4 ] name=ether4-sat rx-flow-control=on
tx-flow-control=on
set [ find default-name=ether5 ] name=ether5-fwa
/interface vrrp
add interface=bridge1 name=vrrp1-LAN priority=200
add interface=ether4-sat name=vrrp2-SAT priority=200 vrid=2
add interface=ether5-fwa name=vrrp3-FWA priority=200 vrid=3
/disk
set sd1 type=hardware
add parent=sd1 partition-number=1 partition-offset=512 partition-size=
“31 266 438 656” type=partition
/interface list
add include=none name=WAN
/port
set 0 name=serial0
/routing table
add disabled=no fib name=to_SAT1
add disabled=no fib name=to_FWA1
/system logging action
set 0 memory-lines=10000
set 3 remote=192.168.10.6
/interface bridge port
add bridge=bridge1 interface=ether1-bbone internal-path-cost=10 path-cost=10
trusted=yes
add bridge=bridge1 interface=dynamic internal-path-cost=10 path-cost=10
trusted=yes
/ip firewall connection tracking
set enabled=yes tcp-close-wait-timeout=1m tcp-established-timeout=1h
tcp-fin-wait-timeout=1m tcp-last-ack-timeout=30s
tcp-syn-received-timeout=1m tcp-syn-sent-timeout=10s
tcp-time-wait-timeout=1m udp-timeout=30s
/ip neighbor discovery-settings
set discover-interface-list=!dynamic
/ipv6 settings
set disable-ipv6=yes max-neighbor-entries=8192
/interface list member
add interface=ether2 list=WAN
add interface=ether4-sat list=WAN
add interface=ether3 list=WAN
/interface ovpn-server server
set auth=sha1,md5
/ip address
add address=192.168.10.18/24 interface=bridge1 network=192.168.10.0
add address=192.168.1.18/24 interface=ether4-sat network=192.168.1.0
add address=192.168.55.18/24 interface=ether5-fwa network=192.168.55.0
add address=192.168.10.20/24 interface=vrrp1-LAN network=192.168.10.0
add address=192.168.1.20/24 interface=vrrp2-SAT network=192.168.1.0
add address=192.168.55.20/24 interface=vrrp3-FWA network=192.168.55.0
add address=192.255.255.18/24 interface=ether2 network=192.255.255.0
/ip cloud
set ddns-enabled=yes ddns-update-interval=5m
/ip dhcp-client
add disabled=yes interface=ether4-sat
/ip dhcp-relay
add dhcp-server=192.168.10.199 disabled=no interface=ether1-bbone name=relay1
/ip dhcp-server alert
add disabled=no interface=ether1-bbone valid-server=DC:2C:6E:6F:61:6E
/ip dns
set allow-remote-requests=yes cache-size=5000KiB max-udp-packet-size=512
servers=8.8.8.8,192.168.10.6
/ip dns static
/ip firewall address-list
add address=127.0.0.1 list=direct
add address=gse.it list=direct
add address=173.249.28.133 list=direct
add address=40.113.176.167 list=direct
add address=54.78.248.218 list=direct
add address=34.252.18.142 list=direct
add address=10.0.0.0/8 list=RFC1918
add address=172.16.0.0/12 list=RFC1918
add address=192.168.0.0/16 list=RFC1918
/ip firewall filter
add action=drop chain=input connection-state=invalid
add action=drop chain=input dst-address=255.255.255.255
add action=drop chain=forward comment=NetBIOS dst-port=137-139,445
in-interface=bridge1 protocol=tcp
add action=drop chain=forward dst-port=137-139,445 in-interface=bridge1
protocol=udp
add action=drop chain=forward dst-address=192.168.10.0/24 src-address=
192.168.50.0/24
add action=drop chain=input dst-port=53 in-interface=vrrp2-SAT protocol=tcp
add action=drop chain=input dst-port=53 in-interface=vrrp3-FWA protocol=tcp
add action=fasttrack-connection chain=forward connection-state=
established,related disabled=yes hw-offload=yes
add action=accept chain=forward connection-state=
established,related,untracked in-interface=all-ethernet
add action=accept chain=forward dst-address=192.168.10.7 dst-port=53
protocol=udp
add action=accept chain=forward dst-address=192.168.10.6 dst-port=53
protocol=udp
add action=accept chain=input in-interface=all-ethernet
add action=accept chain=output out-interface=all-ethernet
/ip firewall mangle
add action=accept chain=prerouting comment=“no PCC ports” dst-port=8883
in-interface=vrrp1-LAN protocol=tcp
add action=accept chain=prerouting comment=“no PCC destinations”
dst-address-list=direct in-interface=vrrp1-LAN
add action=mark-connection chain=input comment=“Sticky connections”
connection-mark=no-mark connection-state=new in-interface=vrrp2-SAT
new-connection-mark=SAT1_conn passthrough=yes
add action=mark-connection chain=input connection-mark=no-mark
connection-state=new in-interface=vrrp3-FWA new-connection-mark=FWA1_conn
passthrough=yes
add action=mark-routing chain=output comment=“Output chain” connection-mark=
SAT1_conn new-routing-mark=to_SAT1 passthrough=yes
add action=mark-routing chain=output connection-mark=FWA1_conn
new-routing-mark=to_FWA1 passthrough=yes
add action=accept chain=prerouting comment=“Accept rule” dst-address-list=
RFC1918 in-interface=vrrp2-SAT
add action=accept chain=prerouting dst-address-list=RFC1918 in-interface=
vrrp3-FWA
add action=accept chain=prerouting dst-address-type=local src-address-type=
local
add action=mark-connection chain=prerouting comment=PCC connection-mark=
no-mark connection-state=new dst-address-type=!local in-interface=
vrrp1-LAN new-connection-mark=SAT1_conn passthrough=no
per-connection-classifier=src-address-and-port:2/0 src-address-type=
!local
add action=mark-connection chain=prerouting connection-mark=no-mark
connection-state=new dst-address-type=!local in-interface=vrrp1-LAN
new-connection-mark=FWA1_conn passthrough=no per-connection-classifier=
src-address-and-port:2/1 src-address-type=!local
add action=mark-routing chain=prerouting connection-mark=SAT1_conn
in-interface=vrrp1-LAN new-routing-mark=to_SAT1 passthrough=yes
add action=mark-routing chain=prerouting connection-mark=FWA1_conn
in-interface=vrrp1-LAN new-routing-mark=to_FWA1 passthrough=yes
/ip firewall nat
add action=masquerade chain=srcnat comment=PCC out-interface=vrrp2-SAT
to-addresses=192.168.1.1
add action=masquerade chain=srcnat out-interface=vrrp3-FWA to-addresses=
192.168.55.1
add action=redirect chain=dstnat comment=DNS dst-address-type=!local
dst-port=53 protocol=udp to-ports=53
add action=redirect chain=dstnat dst-address-type=!local dst-port=53
protocol=tcp to-ports=53
/ip route
add check-gateway=ping disabled=no distance=1 dst-address=0.0.0.0/0 gateway=
192.168.1.1 pref-src=“” routing-table=to_SAT1 scope=30
suppress-hw-offload=no target-scope=10
add check-gateway=ping disabled=no distance=1 dst-address=0.0.0.0/0 gateway=
192.168.55.1 pref-src=“” routing-table=to_FWA1 scope=30
suppress-hw-offload=no target-scope=10
add check-gateway=ping disabled=no distance=1 dst-address=0.0.0.0/0 gateway=
192.168.1.1 pref-src=“” routing-table=main scope=30 suppress-hw-offload=
no target-scope=10
add check-gateway=ping disabled=no distance=2 dst-address=0.0.0.0/0 gateway=
192.168.55.1 pref-src=“” routing-table=main scope=30 suppress-hw-offload=
no target-scope=10
/ip service
set telnet disabled=yes
/ip traffic-flow
set enabled=yes
/ip traffic-flow target
add disabled=yes dst-address=192.168.10.6 port=9996 version=ipfix
/ipv6 firewall filter
add action=drop chain=forward disabled=yes in-interface=bridge1
/snmp
set enabled=yes
/system clock
set time-zone-autodetect=no time-zone-name=Europe/Rome
/system identity
set name=r1-rd
/system logging
add action=remote topics=error
add action=remote topics=critical
add action=remote topics=warning
add action=remote topics=error
add action=remote topics=critical
add action=remote topics=warning
add action=remote topics=info
/system ntp client
set enabled=yes
/system ntp server
set broadcast=yes broadcast-addresses=192.168.10.255 enabled=yes
/system ntp client servers
add address=it.pool.ntp.org
/system scheduler
add disabled=yes interval=1h name=updatehostnames on-event=resolvehostnames
policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon
start-date=1970-01-01 start-time=00:00:00
add disabled=yes interval=23h59m59s name=“queue limiter enable” on-event=
“/queue/simple enable 0,1,2,3,4,5,6” policy=
ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon
start-date=2022-10-11 start-time=07:00:00
add disabled=yes interval=23h59m59s name=“queue limiter disable” on-event=
“/queue/simple disable 0,1,2,3,4,5,6” policy=
ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon
start-date=2022-10-11 start-time=23:00:00
/system script
add dont-require-permissions=no name=resolvehostnames owner=admin policy=
read,write source=“# define variables\r
\n:local list\r
\n:local comment\r
\n:local newip\r
\n:local oldip\r
\n\r
\n# Loop through each entry in the address list.\r
\n:foreach i in=[/ip firewall address-list find] do={\r
\n\r
\n# Get the first five characters of the list name\r
\n :set list [:pick [/ip firewall address-list get $i list] 0 5]\r
\n\r
\n# If they’re ‘host_’, then we’ve got a match - process it\r
\n :if ($list = "host_") do={\r
\n\r
\n# Get the comment for this address list item (this is the host name to u
se)\r
\n :set comment [/ip firewall address-list get $i comment]\r
\n :set oldip [/ip firewall address-list get $i address]\r
\n\r
\n# Resolve it and set the address list entry accordingly.\r
\n : if ($newip != $oldip) do={:set newip [:resolve $comment]\r
\n /ip firewall address-list set $i address=$newip}\r
\n }\r
\n }”
add dont-require-permissions=no name=enable_rt_wan4 owner=admin policy=write
source=“ip route enable [find comment=rt_wan_usb_main]”
add dont-require-permissions=no name=disable_rt_wan4 owner=admin policy=write
source=“ip route disable [find comment=rt_wan_usb_main]”
add dont-require-permissions=no name=vrrp-export-firewall owner=admin policy=
ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon source=“/
ip firewall export file=firewall\r
\n:log info "IP firewall export ok"”
add dont-require-permissions=no name=vrrp-input-firewall owner=admin policy=
ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon source=“/
tool fetch mode=ftp address=192.168.10.11 src-path=ip-firewall.rsc user de
lay 5s\r
\n\r
\n/ip firewall;\r
\n:foreach i in=[find] do={\r
\n/ip firewall;\r
\nremove $i;\r
\n}\r
\n\r
\n/import firewall.rsc”
add dont-require-permissions=no name=“ip connection clear” owner=admin
policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon
source=“/ip firewall connection remove [find]”
/tool graphing interface
add interface=ether1-bbone
add interface=ether2
add interface=ether4-sat
add interface=ether3
/tool graphing queue
/tool graphing resource
add
/tool sniffer

And your /ip firewall filter is half-useless mess, but it’s not breaking it.

Which rules do you think are useless? If needed, no problem in re-writing from scratch (VRRP comes quite useful in this way!).

Thanks again
Luca

Hmm, not sure what you are doing, but whats wrong with 2 or three vlans for subnets, one bridge and then doing PCC as needed.
As well the deviation from default firewall rules ( aka the mess and utter garbage) makes the overall situation far more complex than it needs to be.

Can you state simply without any config or equipment speak what the requirements are.
a.. identify the users/groups of users includiing admin
b. identify what traffic they need.

Wan1 is which ISP type? Is its public IP? Is it static/fixed IP or dynamic?
Wan2 is which ISP type? Is its public IP? Is it static/fixed IP or dynamic?

Any servers involved which would be on the LAN side but independent of PCC.
Did you want to be able to config the router remotely and securely?

Well… indeed it’s intended to be very simple: LAN 192.168.10.0/24 should access internet via PCC, spanned on a Starlink router 192.168.1.1 and a FWA router 192.168.55.1. No port forwarding is needed, neither remote access: simply internet browsing from LAN.
The architecture should be redundant using VRRP.

What do you mean about admin user and groups?

Thanks
Luca

What you mean redundant via VRRP. What is the part you are concerned about??

You have a mickrotik device with two WAN sources. Either both are up, one is up or both are down.
How is VRRP going to help you here??

A network diagram may clear up the mystery.

Ahh. reread the first post, have two HEXES doing VRRP.
Personally not worth the hassle, my hexes have not failed, so not needed IMHO,
But one can always have hex2 kept up to date on hex1 config quite easily.
If it ever fails, then 2 minute substitution.

Addresses on VRRP interfaces should have /32 masks:

/ip address
add address=192.168.10.20/32 interface=vrrp1-LAN
add address=192.168.1.20/32 interface=vrrp2-SAT
add address=192.168.55.20/32 interface=vrrp3-FWA

Then outgoing interfaces will be parent ones, and even though masquerade should have the same effect, it’s better to use srcnat and set VRRP addresses manually:

/ip firewall nat
add action=srcnat chain=srcnat out-interface=ether4-sat to-addresses=192.168.1.20
add action=srcnat chain=srcnat out-interface=ether5-fwa to-addresses=192.168.55.20

Another possibly problematic is src-address-and-port PCC matcher. It uses both uplinks more equally, but different connections to same server will take different paths, and some servers may not like it. It could be avoided by using both-addresses, but traffic distribution can be worse.

As for firewall filter, it doesn’t do much. You’re blocking some ports from LAN, which may not work, because incoming interface is probably going to be vrrp1-LAN and not bridge1. Blocking traffic from .10 subnet to .50 seems like typo, because there’s no .50 elsewhere. Blocking DNS from WAN is generally good idea, but here it’s probably useless, if the router is behind NAT. And in case it isn’t useless (e.g. if there’s public address and NAT 1:1), then you’d want to block udp as well, not just tcp. And the rest after that does nothing, because it’s all accept rules, and anything not matched is accepted anyway. You can check this thread for some ideas.

Thank you!
Indeed the suggested corrections didn’t solve the issue: no browsing.
Anyway I’ll check your kindly mentioned thread to clarify myself.

P.S.: the 192.168.50.0/24 is a guest wifi network, and it shouldn’t access the LAN.

Best
Luca

Hi, VRRP is needed to provide hardware redundancy, and although there was an error (/32 mask), isn’t concerning the problem; the issue is concerning PCC.

Best
Luca

Unfortunately, I don’t see anything obviously wrong that could cause what you were describing. And that description, that’s really weird behaviour.

All VRRP masters are on same router, right?

Yes, correct. The weird matter is that the PCC cascade (isn’t the first one for me) won’t work at all, and by adding a simple accept rule before PCC, thus bypassing it, does work flawlessly, therefore I guess that the overall configuration should be nearly correct, except PCC. I suspect that the reply path from the WAN routers is somewhat messed up.

Best
Luca

I think I see it, the two PCC rules need to have passthrough=yes.

You’re right, my mistake; unfortunately, after correction, same issue. I’ve tried to rewrite the PCC rules several times from scratch, but same problems. Anyway, after your kind suggestions, I’ll redo it again.

Best
Luca

Ok, it seems that the errors were:

  • the usage of VRRP virtual interfaces instead of the local member within firewall/NAT/mangle rules
  • VRRP network /24 instead of /32
  • NAT masquerade replaced by src-nat to IP
  • removed bridge1 (LAN) and used the relevant port instead

Here the working config:

/ip firewall address-list
add address=127.0.0.1 list=direct
add address=gse.it list=direct
add address=173.249.28.133 list=direct
add address=40.113.176.167 list=direct
add address=54.78.248.218 list=direct
add address=34.252.18.142 list=direct
add address=10.0.0.0/8 list=RFC1918
add address=172.16.0.0/12 list=RFC1918
add address=192.168.0.0/16 list=RFC1918
/ip firewall connection tracking
set enabled=yes tcp-close-wait-timeout=1m tcp-established-timeout=1h tcp-fin-wait-timeout=1m tcp-last-ack-timeout=30s tcp-syn-received-timeout=1m tcp-syn-sent-timeout=10s tcp-time-wait-timeout=1m udp-timeout=30s
/ip firewall filter
add action=drop chain=input connection-state=invalid
add action=drop chain=input dst-address=255.255.255.255
add action=drop chain=forward comment=NetBIOS dst-port=137-139,445 in-interface=ether1-bbone protocol=tcp
add action=drop chain=forward dst-port=137-139,445 in-interface=ether1-bbone protocol=udp
add action=drop chain=forward dst-address=192.168.10.0/24 src-address=192.168.50.0/24
add action=drop chain=input dst-port=53 in-interface=ether4-sat protocol=tcp
add action=drop chain=input dst-port=53 in-interface=ether5-fwa protocol=tcp
add action=accept chain=forward connection-state=established,related,untracked in-interface=all-ethernet
add action=accept chain=forward dst-address=192.168.10.7 dst-port=53 protocol=udp
add action=accept chain=forward dst-address=192.168.10.6 dst-port=53 protocol=udp
add action=accept chain=input in-interface=all-ethernet
add action=accept chain=output out-interface=all-ethernet
/ip firewall mangle
add action=accept chain=prerouting comment=“no PCC ports” dst-port=8883 in-interface=ether1-bbone protocol=tcp
add action=accept chain=prerouting comment=“no PCC destinations” dst-address-list=direct in-interface=ether1-bbone
add action=mark-connection chain=input comment=“Sticky connections” connection-mark=no-mark connection-state=new in-interface=ether4-sat new-connection-mark=SAT1_conn passthrough=yes
add action=mark-connection chain=input connection-mark=no-mark connection-state=new in-interface=ether5-fwa new-connection-mark=FWA1_conn passthrough=yes
add action=mark-routing chain=output comment=“Output chain” connection-mark=SAT1_conn new-routing-mark=to_SAT1 passthrough=yes
add action=mark-routing chain=output connection-mark=FWA1_conn new-routing-mark=to_FWA1 passthrough=yes
add action=accept chain=prerouting comment=“Accept rule” dst-address-list=RFC1918 in-interface=ether4-sat
add action=accept chain=prerouting dst-address-list=RFC1918 in-interface=ether5-fwa
add action=mark-connection chain=prerouting comment=PCC connection-state=“” dst-address-type=!local in-interface=ether1-bbone new-connection-mark=SAT1_conn passthrough=yes per-connection-classifier=both-addresses:2/0 src-address-type=!local
add action=mark-connection chain=prerouting connection-state=“” dst-address-type=!local in-interface=ether1-bbone new-connection-mark=FWA1_conn passthrough=yes per-connection-classifier=both-addresses:2/1 src-address-type=!local
add action=mark-routing chain=prerouting connection-mark=SAT1_conn in-interface=ether1-bbone new-routing-mark=to_SAT1 passthrough=yes
add action=mark-routing chain=prerouting connection-mark=FWA1_conn in-interface=ether1-bbone new-routing-mark=to_FWA1 passthrough=yes
/ip firewall nat
add action=src-nat chain=srcnat to-addresses=192.168.1.20
add action=src-nat chain=srcnat to-addresses=192.168.55.20
add action=redirect chain=dstnat dst-address-type=!local dst-port=53 protocol=udp to-ports=53
add action=redirect chain=dstnat dst-address-type=!local dst-port=53 protocol=tcp to-ports=53
/ip route
add check-gateway=ping disabled=no distance=1 dst-address=0.0.0.0/0 gateway=192.168.1.1 pref-src=“” routing-table=to_SAT1 scope=30 suppress-hw-offload=no target-scope=10
add check-gateway=ping disabled=no distance=1 dst-address=0.0.0.0/0 gateway=192.168.55.1 pref-src=“” routing-table=to_FWA1 scope=30 suppress-hw-offload=no target-scope=10
add check-gateway=ping disabled=no distance=1 dst-address=0.0.0.0/0 gateway=192.168.1.1 pref-src=“” routing-table=main scope=30 suppress-hw-offload=no target-scope=10
add check-gateway=ping disabled=no distance=1 dst-address=0.0.0.0/0 gateway=192.168.55.1 pref-src=“” routing-table=main scope=30 suppress-hw-offload=no target-scope=10

Thank you all!
Luca

Sorry to ruin it for you, but no.

If nothing else, now you have two unconditional srcnat rules. So the first one will be used for anything passing through router and nothing will ever get to second one.

So should I set the NAT as the usual way?

/ip firewall nat
add action=masquerade chain=srcnat out-interface=ether4-sat
add action=masquerade chain=srcnat out-interface=ether5-fwa

Or work with masquerade towards an outbound interface list?

Thanks
Luca

I’d use what I wrote before:

/ip firewall nat
add action=srcnat chain=srcnat out-interface=ether4-sat to-addresses=192.168.1.20
add action=srcnat chain=srcnat out-interface=ether5-fwa to-addresses=192.168.55.20

But the main point was that if you thought it was working, it couldn’t. And it’s not just this. If you still have VRRP on LAN (on ether1-bbone instead of previous bridge1), and if devices use VRRP’s IP address (previously 192.168.10.20) as gateway, then in-interface=ether1-bbone can’t work, because in-interface for that traffic is vrrp1-LAN.

Well, I’ve corrected the interfaces as per your kind suggestions.
Now, the problem is how to have the proper output interface selected, related to PCC. A “plain” PCC config can’t select beyond the first output interface, while the output towards a WAN group via masq does some distribution between both, but not in respect to PCC.

Best
Luca