PCC works but no local server access

RouterOS General
1

Based on the guide from https://wiki.mikrotik.com/wiki/Manual:PCC I did create a PCC setup that works fine for internet access, but I lost connectivity to my LAN Servers.

I think I am missing a mangle rule to get it working, would be great if you could help me. I posted my config below.

/ip firewall mangle
add action=accept chain=prerouting comment=ACCEPT_GW1 dst-address=xxx.xxx.xxx.xxx in-interface-list=VLAN
add action=accept chain=prerouting comment=ACCEPT_GW2 dst-address=yyy.yyy.yyy.yyy in-interface-list=VLAN
add action=mark-connection chain=prerouting connection-mark=no-mark in-interface=ether1 new-connection-mark=ISP1_conn
add action=mark-connection chain=prerouting connection-mark=no-mark in-interface=ether6 new-connection-mark=ISP2_conn
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-type=!local in-interface-list=VLAN new-connection-mark=ISP1_conn \
    per-connection-classifier=both-addresses:2/0
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-type=!local in-interface-list=VLAN new-connection-mark=ISP2_conn \
    per-connection-classifier=both-addresses:2/1
add action=mark-routing chain=prerouting connection-mark=ISP1_conn in-interface-list=VLAN new-routing-mark=to_ISP1
add action=mark-routing chain=prerouting connection-mark=ISP2_conn in-interface-list=VLAN new-routing-mark=to_ISP2
add action=mark-routing chain=output connection-mark=ISP1_conn new-routing-mark=to_ISP1
add action=mark-routing chain=output connection-mark=ISP2_conn new-routing-mark=to_ISP2

This is the dhcp client configuration that changes the ip and gateway if I would get a new one from my ISP

/ip dhcp-client
add default-route-distance=32 disabled=no interface=ether6 script="{\r\
    \n\t:if (\$bound=1) do={\r\
    \n\t\t/ip route set [/ip route find where comment=\"0061_WAN6_GW1\"] gateway=\$\"gateway-address\"\r\
    \n\t\t/ip firewall mangle set [/ip firewall mangle find where comment=\"ACCEPT_GW1\"] dst-address=\$\"gateway-address\"\r\
    \n\t\t/ip firewall connection remove [find connection-mark=\"ISP2_conn\"]\r\
    \n\t} \r\
    \n}" use-peer-dns=no
add default-route-distance=33 disabled=no interface=ether1 script="{\r\
    \n\t:if (\$bound=1) do={\r\
    \n\t\t/ip route set [/ip route find where comment=\"0011_WAN1_GW1\"] gateway=\$\"gateway-address\"\r\
    \n\t\t/ip firewall mangle set [/ip firewall mangle find where comment=\"ACCEPT_GW2\"] dst-address=\$\"gateway-address\"\r\
    \n\t\t/ip firewall connection remove [find connection-mark=\"ISP1_conn\"]\r\
    \n\t} \r\
    \n}" use-peer-dns=no

I got some nat rules

/ip firewall nat
add action=dst-nat chain=dstnat dst-port=80 in-interface=ether1 protocol=tcp to-addresses=10.8.20.107 to-ports=80
add action=dst-nat chain=dstnat dst-port=443 in-interface=ether1 protocol=tcp to-addresses=10.8.20.107 to-ports=443
add action=dst-nat chain=dstnat dst-port=5060 in-interface=ether1 protocol=udp to-addresses=10.0.10.253 to-ports=5060
add action=dst-nat chain=dstnat dst-port=15000-30000 in-interface=ether1 protocol=udp to-addresses=10.0.10.253 to-ports=15000-30000
add action=dst-nat chain=dstnat dst-port=32400 in-interface=ether1 protocol=tcp to-addresses=10.8.20.100 to-ports=32400
add action=masquerade chain=srcnat out-interface=ether1
add action=masquerade chain=srcnat out-interface=ether6

And my firewal filter setup

/ip firewall filter
add action=accept chain=input comment="Allow Estab & Related" connection-state=established,related
add action=accept chain=input comment="Allow VLAN" in-interface-list=VLAN
add action=accept chain=input comment="Allow Base_Vlan Full Access" in-interface=BASE_VLAN
add action=drop chain=input comment=Drop
add action=accept chain=forward comment="Allow access to Https on SERVER_VLAN" connection-state=new dst-port=443 in-interface-list=VLAN \
    out-interface=SERVER_VLAN protocol=tcp
add action=accept chain=forward comment="Allow access to Https on SERVER_VLAN" connection-state=new dst-port=443 in-interface-list=WAN \
    out-interface=SERVER_VLAN protocol=tcp
add action=accept chain=forward comment="Allow access to Server on BLUE_VLAN" connection-state=new dst-port=80 in-interface-list=VLAN \
    out-interface=SERVER_VLAN protocol=tcp
add action=accept chain=forward comment="Allow access to Server on BLUE_VLAN" connection-state=new dst-port=80 in-interface-list=WAN out-interface=\
    SERVER_VLAN protocol=tcp
add action=accept chain=forward comment="Allow access to Server on BLUE_VLAN" connection-state=new dst-port=8080 in-interface-list=VLAN \
    out-interface=SERVER_VLAN protocol=tcp
add action=accept chain=forward comment="Allow access to Server on BLUE_VLAN" connection-state=new dst-port=5060 in-interface-list=WAN \
    out-interface=BLUE_VLAN protocol=udp
add action=accept chain=forward comment="Allow access to Server on BLUE_VLAN" connection-state=new dst-port=15000-30000 in-interface-list=WAN \
    out-interface=BLUE_VLAN protocol=udp
add action=accept chain=forward comment="Allow access to Server on BLUE_VLAN" connection-state=new dst-port=500 in-interface-list=WAN \
    out-interface=BLUE_VLAN protocol=udp
add action=accept chain=forward comment="Allow access to Server on BLUE_VLAN" connection-state=new dst-port=4500 in-interface-list=WAN \
    out-interface=BLUE_VLAN protocol=udp
add action=accept chain=forward comment="Allow access to Plex on SERVER_VLAN" connection-state=new dst-port=32400 in-interface-list=WAN \
    out-interface=SERVER_VLAN protocol=tcp
add action=accept chain=forward comment="Allow access to Plex on SERVER_VLAN" connection-state=new dst-port=32400 in-interface-list=VLAN \
    out-interface=SERVER_VLAN protocol=tcp
add action=accept chain=forward comment="Allow access to Https on SERVER_VLAN" connection-state=new dst-port=22 in-interface-list=VLAN \
    out-interface=SERVER_VLAN protocol=tcp
add action=accept chain=forward comment="Allow Estab & Related" connection-state=established,related
add action=accept chain=forward comment="VLAN Internet Access only" connection-state=new in-interface-list=VLAN out-interface-list=WAN
add action=drop chain=forward comment=Drop

Edit: Added DNS settings

/ip dns
set allow-remote-requests=yes servers=192.168.0.1,1.1.1.1,8.8.8.8
/ip dns static
add address=10.8.20.107 name=example.tld
add cname=example.tld name=*.example.tld type=CNAME
2

Create an RFC6890 based address list, then give a !rfc6890 dst-address-list in each of the mark connection mangle rules. Problem solved.

3

Great, thank you, this solved it for me I did:

/ip firewall address-list
add address=10.0.10.0/24 comment=RFC6890 list=rfc6890
add address=10.0.20.0/24 comment=RFC6890 list=rfc6890
add address=10.0.30.0/24 comment=RFC6890 list=rfc6890
add address=10.0.100.0/24 comment=RFC6890 list=rfc6890
add address=10.0.200.0/24 comment=RFC6890 list=rfc6890
add address=10.0.250.0/24 comment=RFC6890 list=rfc6890
add address=10.8.20.0/24 comment=RFC6890 list=rfc6890
add address=192.168.0.0/24 comment=RFC6890 list=rfc6890

And changed the mangle section to:

/ip firewall mangle
add action=accept chain=prerouting comment=ACCEPT_GW1 dst-address=91.138.16.1 in-interface-list=VLAN
add action=accept chain=prerouting comment=ACCEPT_GW2 dst-address=77.74.101.129 in-interface-list=VLAN
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-list=!rfc6890 in-interface=ether1 new-connection-mark=ISP1_conn
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-list=!rfc6890 in-interface=ether6 new-connection-mark=ISP2_conn
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-list=!rfc6890 dst-address-type=!local in-interface-list=VLAN \
    new-connection-mark=ISP1_conn per-connection-classifier=both-addresses:2/0
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-list=!rfc6890 dst-address-type=!local in-interface-list=VLAN \
    new-connection-mark=ISP2_conn per-connection-classifier=both-addresses:2/1
add action=mark-routing chain=prerouting connection-mark=ISP1_conn in-interface-list=VLAN new-routing-mark=to_ISP1
add action=mark-routing chain=prerouting connection-mark=ISP2_conn in-interface-list=VLAN new-routing-mark=to_ISP2
add action=mark-routing chain=output connection-mark=ISP1_conn new-routing-mark=to_ISP1
add action=mark-routing chain=output connection-mark=ISP2_conn new-routing-mark=to_ISP2

Now it works :smiley:

4

Stick the whole 16 subnets from RFC6890, there multi-cast/broadcast subnets too which should be excluded.

Copy-paste from my firewall config to speed it up: https://forum.mikrotik.com/viewtopic.php?f=2&t=172322

5

Ah great thank you. I extended the list and will improve the firewall rules.