Mikrotik PCC Load Balancing affects bonding

Hello friends,

I’ve trying almost about 7 months to solve a problem… I use PCC loadbalancing with classifier set to both addresses and ports but this blocks my l2tp vpn connection on my phone, but i can pass sstp and connect with other MT devices. Is there any solution without change anything on my mangle rules (from both addresses and ports to src address etc).



Thank you very much

Not enough information given. When trying with L2TP from the phone, is the phone connected to LAN WiFi or does it use mobile data and connect via Mikrotik’s WAN? Or does it connect to some remote L2TP/IPsec server via your Mikrotik?

The only case I can imagine where per-connection-classifier could break an L2TP/IPsec connection is one where there is no NAT all the way from the L2TP client to the L2TP server, and the “control” connection (IKE using UDP) is sent via a different WAN than the “data” connection (ESP) - but if there is no NAT, even the fact that different WANs are used should break nothing. If there is NAT, the ESP is encapsulated into UDP and uses the same socket as the control connection so there is no way how per-connection-classifier should break it.

Also, you mention bonding in the topic title but nowhere in the OP, where is bonding in the picture?

Hello Sindy thanks for reply!
My apologies i forgot to say that i have 2 routers involved in this scenario.
1 St MT is acting as loadbalancing
2 nd MT is acting as a PPPoE server Hotspot SSTP Server And l2tp Server

i have NAT in both routers enabled

The mobile phone is trying to connect via mobile data. Please note that if i disable 2 of three WAN’s the l2tp vpn works perfect

The way you put it it sounds like an issue with connection marking so that the L2TP server would respond via the same WAN through which the request from the client came in, but I cannot see why the SSTP clients should work in the same setup. In any case, the next step is described in my automatic signature right below. Both machines, please.

Load Balancer config
/ip firewall filter
add action=accept chain=forward in-interface=pppoe1 protocol=ipsec-esp
add action=accept chain=input connection-nat-state=dstnat connection-state=established,related in-interface-list=WAN
add action=accept chain=forward connection-nat-state=dstnat connection-state=established,related in-interface-list=WAN
add action=drop chain=input connection-nat-state=!dstnat connection-state=invalid,new,untracked in-interface-list=WAN
add action=drop chain=forward connection-nat-state=!dstnat connection-state=invalid,new,untracked in-interface-list=WAN
add action=drop chain=input dst-port=53 in-interface-list=WAN protocol=tcp
add action=drop chain=forward dst-port=53 in-interface-list=WAN protocol=tcp
add action=drop chain=input dst-port=53 in-interface-list=WAN protocol=udp
add action=drop chain=forward dst-port=53 in-interface-list=WAN protocol=udp
/ip firewall mangle
add action=mark-connection chain=input in-interface=pppoe1 new-connection-mark=pppoe1_conn passthrough=yes
add action=mark-connection chain=input in-interface=pppoe2 new-connection-mark=pppoe2_conn passthrough=yes
add action=mark-connection chain=input in-interface=pppoe5 new-connection-mark=pppoe5_conn passthrough=yes
add action=mark-connection chain=input disabled=yes in-interface=pppoe3 new-connection-mark=pppoe3_conn passthrough=yes
add action=mark-connection chain=input disabled=yes in-interface=ether4 new-connection-mark=ether4_conn passthrough=yes
add action=mark-routing chain=output connection-mark=pppoe1_conn new-routing-mark=to_pppoe1 passthrough=yes
add action=mark-routing chain=output connection-mark=pppoe2_conn new-routing-mark=to_pppoe2 passthrough=yes
add action=mark-routing chain=output connection-mark=pppoe5_conn new-routing-mark=to_pppoe5 passthrough=yes
add action=mark-routing chain=output connection-mark=pppoe3_conn disabled=yes new-routing-mark=to_pppoe3 passthrough=yes
add action=mark-routing chain=output connection-mark=ether4_conn disabled=yes new-routing-mark=to_ether4 passthrough=yes
add action=accept chain=prerouting dst-address=192.168.1.0/24 in-interface=sfp1
add action=accept chain=prerouting dst-address=192.168.2.0/24 in-interface=sfp1
add action=accept chain=prerouting dst-address=192.168.3.0/24 in-interface=sfp1
add action=accept chain=prerouting dst-address=192.168.4.0/24 in-interface=sfp1
add action=mark-connection chain=prerouting dst-address-type=!local in-interface=sfp1 new-connection-mark=pppoe1_conn passthrough=yes \
    per-connection-classifier=both-addresses-and-ports:3/0
add action=mark-connection chain=prerouting dst-address-type=!local in-interface=sfp1 new-connection-mark=pppoe2_conn passthrough=yes \
    per-connection-classifier=both-addresses-and-ports:3/1
add action=mark-connection chain=prerouting dst-address-type=!local in-interface=sfp1 new-connection-mark=pppoe5_conn passthrough=yes \
    per-connection-classifier=both-addresses-and-ports:3/2
add action=mark-connection chain=prerouting disabled=yes dst-address-type=!local in-interface=sfp1 new-connection-mark=pppoe3_conn passthrough=yes \
    per-connection-classifier=both-addresses-and-ports:5/2
add action=mark-connection chain=prerouting disabled=yes dst-address-type=!local in-interface=sfp1 new-connection-mark=ether4_conn passthrough=yes \
    per-connection-classifier=both-addresses-and-ports:5/3
add action=mark-routing chain=prerouting connection-mark=pppoe1_conn in-interface=sfp1 new-routing-mark=to_pppoe1 passthrough=yes
add action=mark-routing chain=prerouting connection-mark=pppoe2_conn in-interface=sfp1 new-routing-mark=to_pppoe2 passthrough=yes
add action=mark-routing chain=prerouting connection-mark=pppoe5_conn in-interface=sfp1 new-routing-mark=to_pppoe5 passthrough=yes
add action=mark-routing chain=prerouting connection-mark=pppoe3_conn disabled=yes in-interface=sfp1 new-routing-mark=to_pppoe3 passthrough=yes
add action=mark-routing chain=prerouting connection-mark=ether4_conn disabled=yes in-interface=sfp1 new-routing-mark=to_ether4 passthrough=yes
/ip firewall nat
add action=masquerade chain=srcnat out-interface-list=WAN
add action=dst-nat chain=dstnat disabled=yes dst-port=500,1701,4500 in-interface=pppoe1 protocol=udp to-addresses=10.0.50.2
add action=masquerade chain=srcnat disabled=yes out-interface=all-ppp
add action=masquerade chain=srcnat disabled=yes out-interface=ether2
add action=masquerade chain=srcnat disabled=yes out-interface=ether3
add action=masquerade chain=srcnat disabled=yes out-interface=ether4
add action=masquerade chain=srcnat disabled=yes out-interface=ether5
add action=dst-nat chain=dstnat dst-port=0-65535 in-interface-list=WAN protocol=tcp to-addresses=10.0.50.2
add action=dst-nat chain=dstnat dst-port=0-65535 in-interface-list=WAN protocol=udp to-addresses=10.0.50.2
/ip firewall service-port
set sip disabled=yes





PPPoE,Hotspot,SSTP,L2TP Server Router

/ip firewall address-list
add address=192.168.88.0/22 list=lan
add address=192.168.123.0/24 list=lan
add address=10.100.10.0/24 disabled=yes list=interlan
add address=10.0.4.0/24 list=lan
add address=172.25.20.0/22 disabled=yes list=lan
add address=192.168.100.0/24 list=lan
add address=10.155.15.0/24 list=interlan
add address=172.25.24.0/24 list=lan
add address=10.55.0.0/24 list=lan
add address=192.168.50.0/24 disabled=yes list=interlan
add address=10.1.12.0/24 list=lan
add address=10.19.0.0/30 list=interlan
/ip firewall connection tracking
set enabled=yes tcp-established-timeout=5h
/ip firewall filter
add action=passthrough chain=unused-hs-chain comment="place hotspot rules here" disabled=yes
add action=reject chain=forward disabled=yes layer7-protocol="simole speedtest" reject-with=icmp-host-unreachable
add action=add-src-to-address-list address-list="DNS ATTACK" address-list-timeout=2w chain=input comment="DNS Attack IP>List" dst-port=53 \
    in-interface-list=WAN protocol=udp tcp-flags=""
add action=add-src-to-address-list address-list="DNS ATTACK" address-list-timeout=2w chain=input comment="DNS Attack IP>List" dst-port=53 \
    in-interface-list=WAN protocol=tcp tcp-flags=""
add action=add-src-to-address-list address-list="port scanners" address-list-timeout=2w chain=input comment="Port scanners to list " protocol=tcp psd=\
    21,3s,3,1
add action=add-src-to-address-list address-list="port scanners" address-list-timeout=2w chain=input comment="NMAP NULL scan" protocol=tcp tcp-flags=\
    !fin,!syn,!rst,!psh,!ack,!urg
add action=add-src-to-address-list address-list="port scanners" address-list-timeout=2w chain=input comment="NMAP FIN Stealth scan" protocol=tcp tcp-flags=\
    fin,!syn,!rst,!psh,!ack,!urg
add action=add-src-to-address-list address-list="port scanners" address-list-timeout=2w chain=input comment="SYN/FIN scan" protocol=tcp tcp-flags=fin,syn
add action=add-src-to-address-list address-list="port scanners" address-list-timeout=2w chain=input comment="SYN/RST scan" protocol=tcp tcp-flags=syn,rst
add action=add-src-to-address-list address-list="port scanners" address-list-timeout=2w chain=input comment="FIN/PSH/URG scan" protocol=tcp tcp-flags=\
    fin,psh,urg,!syn,!rst,!ack
add action=add-src-to-address-list address-list="port scanners" address-list-timeout=2w chain=input comment="ALL/ALL scan" protocol=tcp tcp-flags=\
    fin,syn,rst,psh,ack,urg
add action=accept chain=forward comment="Accept Established/Related" connection-state=established,related
add action=accept chain=input comment="accept winbox" disabled=yes dst-port=8291 protocol=tcp
add action=accept chain=input comment="L2TP Accept" dst-port=1701,500,4500 protocol=udp
add action=accept chain=input comment="IPSec Accept" protocol=ipsec-esp
add action=accept chain=input dst-port=445 protocol=tcp
add action=reject chain=forward dst-port=!80,8080 protocol=tcp reject-with=icmp-network-unreachable src-address=172.25.24.1-172.25.24.255
add action=drop chain=forward dst-address=192.168.91.255
add action=drop chain=forward disabled=yes dst-address=192.168.88.0/22 src-address=172.25.20.0/22
add action=drop chain=forward disabled=yes dst-address=192.168.88.0/22 src-address=172.25.24.0/24
add action=drop chain=input comment="dropping DNS Attacks" log-prefix="port scanner blocked" src-address-list="DNS ATTACK"
add action=drop chain=forward comment="dropping DNS Attacks" log-prefix="port scanner blocked" src-address-list="DNS ATTACK"
add action=drop chain=forward comment="Drop Invalid" connection-state=invalid
add action=drop chain=input comment="dropping port scanners" log-prefix="port scanner blocked" src-address-list="port scanners"
add action=drop chain=forward comment="dropping port scanners" log-prefix="port scanner blocked" src-address-list="port scanners"
add action=drop chain=input comment="Drop Attempt Login User" src-address-list=Lst_AttemptLoginIP
add action=drop chain=input comment="Drop Known Hacking IP's" log=yes log-prefix="virus blocked" src-address=216.218.0.0/16
add action=drop chain=input comment="Drop Known Hacking IP's" log=yes log-prefix="virus blocked" src-address=216.218.206.0/24
add action=drop chain=forward comment="Drop Known Hacking IP's" log=yes log-prefix="virus blocked" src-address=216.218.0.0/16
add action=drop chain=forward comment="Drop Known Hacking IP's" log=yes log-prefix="virus blocked" src-address=216.218.206.0/24
add action=drop chain=input log=yes log-prefix="virus blocked" src-address=141.98.0.0/16
add action=drop chain=forward log=yes log-prefix="virus blocked" src-address=141.98.0.0/16
add action=drop chain=input comment=firewall-ADSL connection-nat-state=!dstnat connection-state=new in-interface=ether0
add action=drop chain=forward comment=firewall-ADSL connection-nat-state=!dstnat connection-state=new in-interface=ether0
/ip firewall nat
add action=masquerade chain=srcnat out-interface=ether0
add action=passthrough chain=unused-hs-chain comment="place hotspot rules here" disabled=yes
add action=masquerade chain=srcnat comment="Lan Masquerade" src-address-list=lan
add action=masquerade chain=srcnat comment="MGMT Masquerade" src-address-list=interlan
add action=dst-nat chain=dstnat comment=UniFi dst-address=10.0.50.2 dst-port=8443 protocol=tcp to-addresses=192.168.50.6 to-ports=8443
add action=dst-nat chain=dstnat comment=ubuntu-Speedtest dst-address=10.0.50.2 dst-port=8765 protocol=tcp to-addresses=192.168.50.6 to-ports=8765
add action=dst-nat chain=dstnat comment=dma-Web dst-address=10.0.50.2 dst-port=9980 protocol=tcp to-addresses=192.168.88.4 to-ports=80
add action=dst-nat chain=dstnat comment=UniFi-Inform dst-address=10.0.50.2 dst-port=8080 protocol=tcp to-addresses=192.168.50.6 to-ports=8080
add action=dst-nat chain=dstnat comment=UniFi-STUN dst-address=10.0.50.2 dst-port=3478 protocol=udp to-addresses=192.168.50.6 to-ports=3478
add action=dst-nat chain=dstnat comment=AirControl dst-address=10.0.50.2 dst-port=9082 protocol=tcp to-addresses=192.168.50.6 to-ports=9082
add action=dst-nat chain=dstnat comment="web generator control" dst-address=10.0.50.2 dst-port=5001 protocol=tcp to-addresses=192.168.88.30 to-ports=5001
add action=dst-nat chain=dstnat comment=3CX dst-address=10.0.50.2 dst-port=5005 protocol=tcp to-addresses=192.168.50.254 to-ports=5005
add action=dst-nat chain=dstnat comment=3CX dst-address=10.0.50.2 dst-port=3478 protocol=udp to-addresses=192.168.50.254 to-ports=3478
add action=dst-nat chain=dstnat comment=3CX dst-address=10.0.50.2 dst-port=5090 protocol=udp to-addresses=192.168.50.254 to-ports=5090
add action=dst-nat chain=dstnat comment=UNMS_REM dst-address=10.0.50.2 dst-port=443 protocol=tcp to-addresses=192.168.50.6 to-ports=443
add action=dst-nat chain=dstnat comment=3CX dst-address=10.0.50.2 dst-port=5090 protocol=tcp to-addresses=192.168.50.254 to-ports=5090
add action=redirect chain=dstnat dst-address-type=!local dst-port=53 protocol=udp src-address=172.25.20.3-172.25.23.254 to-ports=53
add action=redirect chain=dstnat comment="payment reminder rule" dst-port=0-65535 protocol=tcp src-address=172.25.24.1-172.25.24.254 to-ports=8080
add action=redirect chain=dstnat comment="disabled account rule" dst-port=0-65535 protocol=tcp src-address=172.25.25.1-172.25.25.254 to-ports=9090
/ip firewall service-port
set sip ports=5060,5061,5062,5063

You have at least two issues there.

First, your per-connection-classifier rules assign a connection-mark regardless whether it already is assigned or not. This causes no issue (except reducing the throughput a tiny bit) for connections initiated from the LAN side because the source and destination addresses and ports don’t change throughout the connection so the same connection-mark value is assigned every time, but it prevents connections initiated from WAN side from “remembering” the WAN through which they came so that the responses would be sent through that same WAN: the connection-mark value gets rewritten when the LAN->WAN response packet is being handled. So add a match condition connection-mark=no-mark to all the per-connection-classifier rules.

Second, you assign connection-mark also in chain=input based on the WAN interface through which the connection came in, but dst-nated packets are not handled by chain=input. So move these rules from chain=input to chain=prerouting as well, and add connection-mark=no-mark to them too, so that also for connections initiated from the internet, only handling of the first packet of each connection causes assignment of connection-mark.

I typically save a bit of CPU throughput by placing a rule action=jump jump-target=mark-conn connection-state=new as the very first one in chain=prerouting of mangle; after that one, the action=accept and action=mark-routing ones follow, and all the connection marking based on various criteria is done in the dedicated chain mark-conn. So only the initial packet of each connection is handled by the dedicated connection marking chain, whereas all the mid-connection packets skip it and go straight to rules translating the already assigned connection marks to routing marks (or not if their destination address is LAN).

And I still don’t get how it is possible that SSTP clients can connect successfully through WAN with the current setup.

sindy plese confirm me that you mean to change my rules from inputo to prerouting…

Thanks

Yes, but only those in mangle with action=mark-connection. And don’t forget to add connection-mark=no-mark to all rules with action=mark-connection.

Super !!! Thank you very much
Is working like a charm!