Hi,
A few days ago I set VLANs up in my physical LAN, then I thought that it was time to add more features like Failover/Load Balancing and Policy Based Routing.
To avoid to messing up with my LAN, I’m testing the setup in a virtual environment (GNS3), so you won’t see any wireless interface, and I came up with this:
# 2024-05-07 13:02:25 by RouterOS 7.14.3
# software id =
#
/interface bridge
add name=bridge1 vlan-filtering=yes
/interface ethernet
set [ find default-name=ether1 ] name=ether1-WAN1
set [ find default-name=ether2 ] name=ether2-WAN2
set [ find default-name=ether3 ] comment="My PC" name=ether3-access10
set [ find default-name=ether4 ] comment=Server name=ether4-access20
set [ find default-name=ether5 ] name=ether5-access30
/interface vlan
add interface=bridge1 name=vlan10-main vlan-id=10
add interface=bridge1 name=vlan20-server vlan-id=20
add interface=bridge1 name=vlan30-guest vlan-id=30
/disk
set slot1 slot=slot1
set slot2 slot=slot2
set slot3 slot=slot3
set slot4 slot=slot4
set slot5 slot=slot5
set slot6 slot=slot6
set slot7 slot=slot7
set slot8 slot=slot8
set slot9 slot=slot9
/interface list
add name=WAN
add name=LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip pool
add name=dhcp_pool1 ranges=172.16.10.100-172.16.10.254
add name=dhcp_pool2 ranges=172.16.20.100-172.16.20.254
add name=dhcp_pool3 ranges=172.16.30.100-172.16.30.254
/ip dhcp-server
add address-pool=dhcp_pool1 interface=vlan10-main name=dhcp-vlan10
add address-pool=dhcp_pool2 interface=vlan20-server name=dhcp-vlan20
add address-pool=dhcp_pool3 interface=vlan30-guest name=dhcp-vlan30
/port
set 0 name=serial0
/queue simple
add burst-limit=7M/10M burst-threshold=7M/10M burst-time=10s/10s max-limit=\
5M/8M name=GUEST-Subnet target=172.16.30.0/24
/routing table
add disabled=no fib name=to-isp1
add disabled=no fib name=to-isp2
/interface bridge port
add bridge=bridge1 frame-types=admit-only-untagged-and-priority-tagged \
interface=ether3-access10 pvid=10
add bridge=bridge1 frame-types=admit-only-untagged-and-priority-tagged \
interface=ether4-access20 pvid=20
add bridge=bridge1 frame-types=admit-only-untagged-and-priority-tagged \
interface=ether5-access30 pvid=30
/ipv6 settings
set disable-ipv6=yes
/interface bridge vlan
add bridge=bridge1 tagged=bridge1 untagged=ether3-access10 vlan-ids=10
add bridge=bridge1 tagged=bridge1 untagged=ether4-access20 vlan-ids=20
add bridge=bridge1 tagged=bridge1 untagged=ether5-access30 vlan-ids=30
/interface list member
add interface=ether1-WAN1 list=WAN
add interface=ether2-WAN2 list=WAN
add interface=vlan10-main list=LAN
add interface=vlan20-server list=LAN
add interface=vlan30-guest list=LAN
/ip address
add address=172.16.10.1/24 interface=vlan10-main network=172.16.10.0
add address=172.16.20.1/24 interface=vlan20-server network=172.16.20.0
add address=172.16.30.1/24 interface=vlan30-guest network=172.16.30.0
/ip dhcp-client
add add-default-route=no interface=ether1-WAN1 script="# APPLY TO /ip dhcp-cli\
ent add script= section.\r\
\n\r\
\n:global HealthCheckIP 8.8.4.4 ; # IP to use to check if ISP pat\
h is working. Use different IPs for each ISP.\r\
\n:global ISPPriority 1 ; # Which ISP path to use first. \
1 is the highest priority. Each ISP needs a different priority value.\r\
\n:global ISPName \"ISP_\$interface\" ; # This creates the comments \
for the routes and is used to find and change/delete the entries.\r\
\n\r\
\n# Make sure to set Add Default Route to \"no\" on the DHCP Client.\r\
\n/ip dhcp-client\r\
\nset [find where interface=\$interface] add-default-route=no\r\
\n\r\
\n/ip route\r\
\n\r\
\n# Add Recursive Gateway Health Check IP Monitor\r\
\n:local count [:len [find where comment=\"\$ISPName_Monitor\"]]\r\
\n:if (\$bound = 1) do={\r\
\n :if (\$count = 0) do={\r\
\n add comment=\"\$ISPName_Monitor\" disabled=no distance=1 dst-add\
ress=\"\$HealthCheckIP/32\" \\\r\
\n gateway=\$\"gateway-address\" scope=10 target-scope=10\r\
\n } else={\r\
\n :if (\$count = 1) do={\r\
\n :local test [find where comment=\"\$ISPName_Monitor\"]\r\
\n :if ([get \$test gateway] != \$\"gateway-address\") do={\r\
\n set \$test gateway=\$\"gateway-address\"\r\
\n }\r\
\n } else={\r\
\n :error \"Multiple routes found\"\r\
\n }\r\
\n }\r\
\n} else={\r\
\n remove [find where comment=\"\$ISPName_Monitor\"]\r\
\n}\r\
\n\r\
\n# Add 0.0.0.0/0 route to ISP Gateway\r\
\n:local count2 [:len [find where comment=\$ISPName]]\r\
\n:if (\$bound = 1) do={\r\
\n :if (\$count2 = 0) do={\r\
\n add check-gateway=ping comment=\$ISPName disabled=no distance=\$\
ISPPriority dst-address=0.0.0.0/0 \\\r\
\n gateway=\$HealthCheckIP scope=30 target-scope=11\r\
\n } \r\
\n} else={\r\
\n remove [find where comment=\$ISPName]\r\
\n}"
add add-default-route=no interface=ether2-WAN2 script="# APPLY TO /ip dhcp-cli\
ent add script= section.\r\
\n\r\
\n:global HealthCheckIP 1.0.0.1 ; # IP to use to check if ISP pat\
h is working. Use different IPs for each ISP.\r\
\n:global ISPPriority 1 ; # Which ISP path to use first. \
1 is the highest priority. Each ISP needs a different priority value.\r\
\n:global ISPName \"ISP_\$interface\" ; # This creates the comments \
for the routes and is used to find and change/delete the entries.\r\
\n\r\
\n# Make sure to set Add Default Route to \"no\" on the DHCP Client.\r\
\n/ip dhcp-client\r\
\nset [find where interface=\$interface] add-default-route=no\r\
\n\r\
\n/ip route\r\
\n\r\
\n# Add Recursive Gateway Health Check IP Monitor\r\
\n:local count [:len [find where comment=\"\$ISPName_Monitor\"]]\r\
\n:if (\$bound = 1) do={\r\
\n :if (\$count = 0) do={\r\
\n add comment=\"\$ISPName_Monitor\" disabled=no distance=1 dst-add\
ress=\"\$HealthCheckIP/32\" \\\r\
\n gateway=\$\"gateway-address\" scope=10 target-scope=10\r\
\n } else={\r\
\n :if (\$count = 1) do={\r\
\n :local test [find where comment=\"\$ISPName_Monitor\"]\r\
\n :if ([get \$test gateway] != \$\"gateway-address\") do={\r\
\n set \$test gateway=\$\"gateway-address\"\r\
\n }\r\
\n } else={\r\
\n :error \"Multiple routes found\"\r\
\n }\r\
\n }\r\
\n} else={\r\
\n remove [find where comment=\"\$ISPName_Monitor\"]\r\
\n}\r\
\n\r\
\n# Add 0.0.0.0/0 route to ISP Gateway\r\
\n:local count2 [:len [find where comment=\$ISPName]]\r\
\n:if (\$bound = 1) do={\r\
\n :if (\$count2 = 0) do={\r\
\n add check-gateway=ping comment=\$ISPName disabled=no distance=\$\
ISPPriority dst-address=0.0.0.0/0 \\\r\
\n gateway=\$HealthCheckIP scope=30 target-scope=11\r\
\n } \r\
\n} else={\r\
\n remove [find where comment=\$ISPName]\r\
\n}"
/ip dhcp-server lease
add address=172.16.10.100 client-id=1:0:c:29:c3:55:4b mac-address=\
00:0C:29:C3:55:4B server=dhcp-vlan10
/ip dhcp-server network
add address=172.16.10.0/24 dns-server=172.16.10.1 gateway=172.16.10.1
add address=172.16.20.0/24 dns-server=172.16.10.1 gateway=172.16.20.1
add address=172.16.30.0/24 dns-server=172.16.10.1 gateway=172.16.30.1
/ip dns
set allow-remote-requests=yes servers=1.1.1.1,8.8.8.8
/ip firewall address-list
add address=192.168.0.0/16 list=Private-IPs
add address=172.16.0.0/12 list=Private-IPs
add address=172.16.10.100 list=Clients-ISP1
/ip firewall filter
add action=accept chain=input dst-port=8291 in-interface=vlan10-main \
protocol=tcp
add action=accept chain=input comment=\
"defconf: accept established,related,untracked" connection-state=\
established,related
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment=\
"defconf: accept to local loopback (for CAPsMAN)" dst-address=127.0.0.1
add action=accept chain=input comment="Allow LAN DNS queries-UDP" dst-port=53 \
in-interface-list=LAN protocol=udp
add action=accept chain=input comment="Allow LAN DNS queries - TCP" dst-port=\
53 in-interface-list=LAN protocol=tcp
add action=reject chain=input in-interface-list=LAN log=yes log-prefix=\
rej_LAN reject-with=icmp-admin-prohibited
add action=drop chain=input comment="drop all else"
add action=accept chain=forward comment="defconf: accept in ipsec policy" \
ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" \
ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" \
connection-state=established,related hw-offload=yes
add action=accept chain=forward comment=\
"defconf: accept established,related, untracked" connection-state=\
established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" \
connection-state=invalid
add action=accept chain=forward in-interface=vlan10-main out-interface-list=\
LAN
add action=accept chain=forward comment="allow internet traffic" \
in-interface-list=LAN out-interface-list=WAN
add action=accept chain=forward comment=\
"allow dst-nat from both WAN and LAN (including port forwarding)" \
connection-nat-state=dstnat
add action=drop chain=forward comment="drop all else"
/ip firewall mangle
add action=mark-routing chain=prerouting comment="PBR to isp1" \
dst-address-list=!Private-IPs in-interface=vlan30-guest new-routing-mark=\
to-isp1 passthrough=no
add action=mark-routing chain=prerouting comment="PBR to isp2" \
dst-address-list=!Private-IPs in-interface=vlan10-main new-routing-mark=\
to-isp2 passthrough=no src-address-list=Clients-ISP1
/ip firewall nat
add action=masquerade chain=srcnat out-interface-list=WAN
add action=dst-nat chain=dstnat dst-address=!172.16.10.1 dst-port=53 log=yes \
protocol=udp src-address=!172.16.10.1 to-addresses=172.16.10.1 to-ports=\
53
add action=dst-nat chain=dstnat dst-address=!172.16.10.1 dst-port=53 log=yes \
protocol=tcp src-address=!172.16.10.1 to-addresses=172.16.10.1 to-ports=\
53
/ip firewall service-port
set ftp disabled=yes
set tftp disabled=yes
/ip route
add comment=ISP_ether2-WAN2_Monitor disabled=no distance=1 dst-address=\
1.0.0.1/32 gateway=10.2.2.1 scope=10 target-scope=10
add check-gateway=ping comment=ISP_ether2-WAN2 disabled=no distance=1 \
dst-address=0.0.0.0/0 gateway=1.0.0.1 pref-src="" routing-table=to-isp2 \
scope=30 suppress-hw-offload=no target-scope=11
add comment=ISP_ether1-WAN1_Monitor disabled=no distance=1 dst-address=\
8.8.4.4/32 gateway=10.1.1.1 scope=10 target-scope=10
add check-gateway=ping comment=ISP_ether1-WAN1 disabled=no distance=1 \
dst-address=0.0.0.0/0 gateway=8.8.4.4 pref-src="" routing-table=to-isp1 \
scope=30 suppress-hw-offload=no target-scope=11
/ip service
set telnet disabled=yes
set ftp disabled=yes
set www disabled=yes
set ssh disabled=yes
set api disabled=yes
set api-ssl disabled=yes
/system clock
set time-zone-name=Europe/Paris
/system identity
set name=MikroTik-Lab
/system note
set show-at-login=no
/tool mac-server
set allowed-interface-list=none
/tool romon
set enabled=yes
In a few words, VLAN30(guest) must always go via ISP1 if available, and one IP (at the moment) on VLAN10(main) via ISP2 if available.
Since the 2 WAN IPs are dynamic, I copied a script to their DHCP-clients which automatically creates routes. I also modified the old recursive failover setup and set distance 1 on both for load balancing. Everything seems to be working as expected so far apparently. However, I realized that if one ISP goes down, the VLANs do not switch to the other one that is still up now. While I was looking up a reliable solution I came across this video:
from min: 27:00, he said you need to add two other routes to accomplish that, but that setup would work only for static WAN IPs which is not my case.
So, I am now trying to figure out how I can make load balancing/failover work along with PBR.
I should probably modify the scripts in their dhcp clients, but I am anything but good with Mikrotik scripts.
Could you please help me?
Thanks
Just looking at your etherports on the config, I get confused because your diagram and your wording are in conflict. Without consistency, there is no point in assessing config.
The ports on your router
ether1 and ether2 Go to WAN – config text – Check
ether3 goes to Switch1 – config text— WRONG, says goes to PC access port 10
ether4 goes to Switch3 – config text – WRONG, says goes to Server access port 20
ether5 goes to Switch2 – config text – WRONG, says goes to access port30
Not my problem your diagram does not match the first few lines of the config.
Stopped me cold. Not going to waste my time on such blatant inconsistencies.
If you had said ignore the etherport designations on the diagram, because they were wrong, then I would have been prepared
Your response did not clarify the discrepancy.
I will be moving on.
Also this is not your first post, please use code tags for configs especially when long.
That is one approach get rid of the offending diagram LOL.
(1) Since you are queueing and mangling, we need to remove the fastrack rule from the forward chain.
(2) Primary WAN for vlan30 is WAN1
(3) Primary WAN for one user on vlan10 is WAN2 → 172.16.10.100
(4) All other users should share the available two WANS as equally as possible.
(5) I cannot guarantee your scripts are correct, ( I keep it simple )
NOW that we have the Routing and primary/secondary and recursive out of the way. The router knows how to deal with traffic without LB.
Now we introduce the LB. We also have to consider that you want VLAN30 to go out WAN2, and one person in VLAN10 to go out WAN1.
/ip firewall address-list
add address=172.16.10.0/24 list=MySubnets
add address=172.16.20.0/24 list=MySubnets
add address=172.16.30.0/24 list=MySubnets /ip firewall mangle
{ first we need to ensure that required traffic is handled prior to load balancing, thus order is important } add action=accept chain=prerouting in-interface-list=LAN dst-address-type=local comment=“allows local traffic to router”
add action=accept chain=prerouting in-interface-list=LAN dst-address-MySubnets list= comment=“allows local to local traffic on router”
add action=mark-route chain=prerouting src-address=172.16.30.0/24 new-routing-mark=use-isp1 passthrough=no comment=“send vlan30 to ISP1”
add action=mark-route chain=prerouting src-address=172.168.10.100 new-routing-mark=use-isp2 passthrough=no comment=“send user to ISP2”[/i]
I could have gotten cute and used the fact that we made WAN1 primary ( as then vlan30 users would automatically go there ).
However I chose simply to create the two WANS as is and use mangling to ensure connectivity as required.
Dont be caught up on the fact that one wan is primary and another secondary. This is needed for normal routing and setup of routes regardless of anything else.
IF there were three WANS it would be the same wan1 dist=1 wan2=dist=3 wan3-dist=3
Once the main routes have been made then we establish any other specific requirements.
In your case
a. failover → methodology used recursive combined with check-gateway=Ping ( some use netwatch )
b. Specific users go out specific WANS - two options here, easiest is ROUTING RULES, however, since you also wanted LB, which does require mangling it made sense to go all mangling.
Reason being is manging takes precedence over ROUTING RULES, so we opted to do all finessing via mangles.
c. LBalancing requires mangling.
Finally, to the point of the IP routes and the IP DHCP clients, I simply used WORDING that was unique on the IP Routes comments, so that they could be clearly added to the IP DHCP Client Scripts.
I separated Primary to identify the main table route requiring updating and Alternate to identify the new routing-mark/table we used during LB.
Truth be told it could look like this and still work fine.
SCRIPT BOX ENTRY IN IP DHCP CLIENT: ISP1 :if ($bound=1) do={
:local gw $“gateway-address”
/ip route set [ find comment=“BelgianChocolate” gateway!=$gw ] gateway=$gw
}
SCRIPT BOX ENTRY IN IP DCHP CLIENT: ISP2 :if ($bound=1) do={
:local gw $“gateway-address”
/ip route set [ find comment=“CanadianMapleSyrup” gateway!=$gw ] gateway=$gw
}
In other words, I didnt need to create Primary and Alternate I could have used one single comment as the affected variable is the same within each routing table for ISP1 and ISP2.
I separated them out to make it clear to you that we were keeping the gatewayIP up to date for BOTH routes of each ISP. Also for logging purposes, not sure if would make a difference for troubleshooting. If it was me I would go the leaner router shown above.
Correct, The only difference without LBing would be the mangling for LBing.
Remember the routes we creates were also to ensure some users went out wan1 and some out wan2 along with establishing orderly main table routes.
Without loadbalancing we would still need all six routes.
I would have routing rules instead of mangling to ensure users went to right WAN before failing over to the other WAN.
So without LB I would not have mangled.
Now as soon as you introduce either servers on the LAN or VPN terminating on the router, then mangling would come back.
+++++++++++++++++++++++
Looking at the diagram, as required, WAN1 is primary being available and selected as the main route of choice (black), while wan2 is available but not selected ( blue )
Both special table routes are available for use (black)
What is weird is why the first two have a comment of ISP1 ?? SHould be ISP1, and ISP2
It seemed that there were too many routes to me. So I set add default routeNO on dhcp-clients
Failover still works nonetheless.
I still have to set LB and mangle rules
No one can reach internet anymore.
If I disable Mangle rules and the “AlternateWAN” routes they can, but there is no PBR of course
Anyway an “export” is worth a thousand words
There is no shash symbol before the Primary and Alternate comments. It must be something added by the export command.
Instead, if there is no slash symbol when you run the script it doesn’t work.
I also fixed the comment AlternateW2 in route that didn’t match; still same problem anyway.
Maybe the mangle rules need a double check.
Thanks
After I setup PBR and failover I realized that the router itself couldn’t reach the internet, so I had to add two more routes with a simple failover (not recursive; some problems with it) set on the main table.
Please check if you see something wrong with it.
Thanks
Okay after reading that you didnt have routes, and needed to add two, I figured out what was wrong
You need to go back to IP DHCP client.
on DHCP tab select default route=YES
on Advanced tab put in default route of 255
The script remains the same in the advanced tab.
With that it will work and you can remove the extra routes you put in.
