@sk0003
I would like to compare that with mine; how do you ‘export’ the config you are showing me? That ‘Code All’?
In Winbox, go to New Terminal and type in /export file=namefile.rsc
Then go to Files in the menu and right click the file that was created and Download to your PC. Then open that with a text editor.
Complete instructions here:
http://forum.mikrotik.com/t/forum-rules/173010/1
@jaclaz: Thanks, RTFM ![]()
Okay, I feel that the problem is that I have a Bridge. None of that was mentioned in any of the videos, and I have no idea what effect this has.
In this version, I have not disabled the IP address of 192.168.188.1, done that as well but made no difference (and I also don’t see why it would need to be disabled?)
… Update: I saw that you have a Bridge too, and it is disabled. I just disabled my Bridge, and lost all access, and now have to reboot to factory settings yet again…
This is what I have/had:
# 2024-12-25 07:17:01 by RouterOS 7.15.1
# software id = NH64-EMY2
#
# model = SXTR
/interface bridge
add admin-mac=F4:1E:57:22:D7:B9 auto-mac=no comment=defconf name=bridge
/interface lte
set [ find default-name=lte1 ] allow-roaming=no band="" sms-read=no
/interface vlan
add interface=ether1 name=man vlan-id=2
add interface=ether1 name=net vlan-id=3
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
/interface lte apn
set [ find default=yes ] passthrough-interface=net passthrough-mac=auto
/ip pool
add name=default-dhcp ranges=192.168.188.10-192.168.188.254
/queue type
add fq-codel-ecn=no kind=fq-codel name=fq-codel-ethernet-default
/queue interface
set ether1 queue=fq-codel-ethernet-default
set ether2 queue=fq-codel-ethernet-default
/interface bridge port
add bridge=bridge comment=defconf interface=ether1
add bridge=bridge comment=defconf interface=ether2
/ip neighbor discovery-settings
set discover-interface-list=LAN
/interface list member
add comment=defconf interface=bridge list=LAN
add comment=defconf interface=lte1 list=WAN
/ip address
add address=192.168.188.1/24 comment=defconf interface=bridge network=\
192.168.188.0
/ip dhcp-server
add address-pool=default-dhcp disabled=yes interface=bridge name=defconf
/ip dhcp-server network
add address=192.168.188.0/24 comment=defconf dns-server=192.168.188.1 \
gateway=192.168.188.1 netmask=24
/ip dns
set allow-remote-requests=yes
/ip dns static
add address=192.168.188.1 comment=defconf name=router.lan
Not sure if you are still around, but in the end after days of endless factory resets, I ended up with roughly what you proposed.
I never ever got a VLAN to work, either with the SXT in default Bridge mode, or in passthrough.
So now I left the SXT LTE6 as default, except I closed the DHCP server. Then I added the IP address manually from the Bridge to the primary /24 network, as 192.168.50.253.
. 254 is the Hex main router, connected to the Starlink WAN.
Then on the Hex I added a second route 0.0.0.0/0 to 192.168.50.253 distance 2.
When I now pull the plug on the Starlink WAN, it re-routes to the SXT okay.
But when I put it back in, some bad stuff happens.
- I lose all ICMP connections from the Windows PC’s to the Internet. However, other TCP connections work. UDP, don’t know. To fix this, I need to restart the Ethernet interfaces on all Windows PC’s.
This problem does NOT occur on Android devices.
A remote connection to the remote Windows PC in the shed in the paddock fails; I then have to also reset my reverse-SSH pipe system on a local Raspberry-PI router (which allows TCP access by Port to equipment on my local LAN via a VPS I have) .
So while I do have a working secondary Internet system, there are problems which I do not fully understand.
Also, I will need a script which can enable/disable the two Internet links based on a ‘ping’ system, but in such a way that it does not switch over on any small packet loss, which happens on Starlink quite a bit.
Any ideas welcome, and any info on the ICMP Ping issues also…
Not @Amm0 but if I may, these are the issues I’ve mentioned a few posts ago. The following is an assumption that needs to be confirmed, but: echo requests (pings) have an ID which stays the same as long as the particular ping command is running, so the firewall treats them (and the responses to them that bear the same ID) as a single logical connection. When you start pinging while the Starlink uplink is down, the first ping request gets to the hEX which forwards it via the SXT, so no src-nat is activated for the connection. It also should send back a notification about the .253 (the SXT) being a better router than the .254 (itself) to the sender, so the sender should send the subsequent echo request packets directly via SXT, bypassing routing on the hEX, which the Androids probably do and the Windows probably don’t. So the tracked connection in the hEX dies off within 10 seconds as it does not get updated by subsequent packets in the Android case, but stays alive as it keeps getting updated in the Windows case. Now once the Starlink path becomes available again, the incoming request packets keep hitting the existing tracked connection without src-nat activated but start getting routed out via Starlink; the Starlink router has no dedicated route towards 192.168.50.0/24 so even if it src-nats the requests to its WAN address (from the CGNAT range) and thus receives responses to them, after un-src-nating the responses their destination address becomes 192.168.5.x so the Starlink router sends them back to the sky.
I usually don’t care about pings since whatever daemon uses them for checking address or path availability doesn’t send all of them with the same ID, but I do care about UDP in this regard, so I periodically run a housekeeping script that removes tracked connections whose reply-dst-address does not match the one of currently active out-interface.
As for TCP connections, they give up within tens of seconds, and either the application automatically initiates a new connection from a different local port so it gets treated as a separate tracked connection, or it doesn’t (like SSH) and you have to do that manually.
All the above would work almost the same even if the SXT was connected using another subnet. Just instead of no src-nat at all, the connections that initially established via SXT would remeber a wrong reply-dst-address.
Thanks for that; I think the fact that I also seem to lose my remote control (called “Remote Utilities”) is likely because it will use UDP.
So based on what you describe, what can I do to clear the situation after a return to the Starlink WAN?
Just to confirm, when I pull the plug on the Starlink WAN, the fallback to the SXT is immediate and works perfectly for most connected equipment, but it goes wrong on return to the Starlink.
Is there a command / script which I can use to force/clear the previous connection, like your housecleaning script?
Unfortunately, all the devices where I have this script are currently offline, so I cannot copy-paste it.
So the following is not tested:
:if ([:len [/ip route find where dst-address=0.0.0.0/0 active distance=1]]>0) do={
:foreach conn in=[/ip firewall connection find where !srcnat !(dst-address~"^(192\\.168|10\\.|172\\.(1[6-9]|2|3[01])\\.)")] do={
:do command={/ip firewall connection remove $conn} on-error={:nothing}
}
}
**EDIT:**fixed a missing !
Only as a side-side note, I remember that rextended advised to use a “remaining time” filter of 60 seconds to avoid invalid results when removing connections:
http://forum.mikrotik.com/t/delete-all-connection-in-firewall-connections/94436/1
though it is aimed to more complex setups with lots of existing connections, while this should match only a handful of connections, and thus should be very fast in execution, a shorter time like 10 seconds may still be advisable to avoid issues.
Only needed if you consistently get issues like " no such item (4)" when running the script by sindy.
@rextended is a very meticulous guy, and my script actually follows his other solution to the same issue, i.e. to use a foreach cycle rather than applying the remove command on the whole list, and to make sure that a failed attempt to remove an already non-existent individual item will not break the whole script by using the :do command on-error construct.
Ahh, good to know, thanks
, the trick is the “on error” , I didn’t know that it actually allowed the prosecution of the loop in case of error. ![]()
@BartKindtNZ - sorry just reading this. But @sindy offers better advice here. I put him in the “meticulous” category too
.
I was trying to get you up-and-running in few steps to be able to test/tweak… but I forgot that that new LTE devices use 192.168.188.1 as their default, which makes my quick routing+VRR{ suggestion slightly different. Basically I just a fan separating problems. i.e. make sure LTE work on SXT first, then move on to adding to hEX [either via routing or LTE passthrough], and then finally optimizing failover.
But on checking starlink… the check-gateway=ping method should be okay. It’s actually 3 pings in a row that have to fail, so takes a 30 second outage to failover to LTE. I’m not a fan of the “recursive routes” (i.e. adding routes with 8.8.8.8/etc) - at least to start – since it’s very exacting config to get right. You can see in starlink app the length of longest outage, and if less than 30 seconds, the basic method should be okay.
And to expand what @sindy points out the /ip/firewall/connection is what will effect recovery after “switching WANs”. /ip/firewall/connections essentially “caches” connections - so if it remembers a connection it will send it out the dead WAN until it times out (or clear by script like one above). Stuff like TCP from web request recover by retry typically. But UDP does not have “connections”, so /ip/firewall/connections has rules to “guess”* at UDP connections (*there are setting to control) so recovery speed for UDP depends on how the UDP-based application behaves - but it’s typically UDP things that require scripting to clear connections if you want a faster failover.
I am strugling to understand the IP address structure in the script; I am a Delphi Pascal programmer, and this script with all the escape characters etc. I find difficult to decode. Before I use it I first must check if the IP addresses are correct for my setup. That is: Default gataway is 192.168.1.1 ; Backup gateway is at 192.168.50.253. I will have to study the scripting language to make sense out of it.
Big thanks to all who are helping!
The IP addresses are all the private ones (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16) and the dst-address attribute of a tracked connection is typically a socket (ip.add.re.ss:port), so the only possible way of matching is a regular expression. In regular expressions, a dot represents “any symbol”, so it has to be escaped by a backslash in order to be interpreted literally (unless I have it wrong), but since it is within a string, you have to escape the backslash by another one so that a single backslash would make it to the regexp ![]()
So you can simplify that provided that all the private addresses you use in your LAN come from just one of the three RFC1918 ranges, but it’s a forum so I prefer the examples to be universally applicable.
So in plain language:
if the route via Starlink is active {
for each item (conn) on a list of tracked connections that are not src-nated and their destination address is not a private one {
remove the item from the connection tracking list, ignoring eventual errors
}
}
Now the above is a great example of “rubber duck debugging”, because as translating the script into plain English, I’ve noticed that while debugging the regular expression on my router, I removed the ! before the (dst-address …) and forgot to put it back, so I’m editing that post now.
Big thanks for all that; Now the stupid question: Where do I insert this script?
On the command line:
/system script add name=housekeeper
/system script edit housekeeper source
paste the script copied from my edited post above and press Ctrl-O to save it.
Then imitate the outage of Starlink and its recovery, and use /system script run housekeeper - if that helps the strayed connections to start working again, it makes sense to add a /tool scheduler item that will spawn it every minute or so.
@sindy
That works brilliantly, thanks!
Now the last thing I need to sort out, is how to drop the Starlink WAN interface when it loses Internet connection. I can pull the plug on the Starlink WAN, and it switches over perfectly, but at the moment I do not yet have a test/ping to 8.8.8.8 which will then cause the STARLink WAN to drop.
Would the script from sk0003 work in my case?
Big thanks for all your help!
For detection of failure of starlink… Three+ choices:
- “check-gateway=ping”, see https://help.mikrotik.com/docs/spaces/ROS/pages/4390920/Load+Balancing#LoadBalancing-SimpleFailoverExample & post my above
- “Recursive Routes”, see https://help.mikrotik.com/docs/spaces/ROS/pages/26476608/Failover+WAN+Backup & @sk0003 series of posts, including the update script in /ip/dhcp-client:
-
Netwatch scripts, see https://help.mikrotik.com/docs/spaces/ROS/pages/8323208/Netwatch#Netwatch-QuickExample … which we don’t cover in this thread but this requires custom scripts but netwatch allows a very flexible approach to detecting failures.
-
BFD, see https://help.mikrotik.com/docs/spaces/ROS/pages/191299691/BFD … again not covered here, but it requires BGP and not usable in home/SMB situations, but its the “fastest” at detecting failure
Pick your poison. Also note, you only need it on the starlink, since LTE will drop and is the “last choice” anyway.
If it can be adapted to your situation (and it seems to me it can), this:
http://forum.mikrotik.com/t/simpler-failover-for-two-gateways-i-found-working/169108/1
further simplified:
http://forum.mikrotik.com/t/simpler-failover-for-two-gateways-i-found-working/169108/1
is IMHO the simplest method (using a simple Netwatch script that just enables and disables a route).
I have read here and there that a Starlink connection can be problematic because it can disconnect temporarily for a few seconds several times a day to soon resume by itself, so I believe the timing of the failover method is important and a netwatch script (more complex than the one in the given example) or a “fine tuning” of the ICMP probe settings might be more suitable than the check gateway or recursive.
In my understanding, “check-gateway=ping” and “recursive next-hop search” mostly only make sense together. Using netwatch is fine as long as you use only a single canary IP per WAN; as soon as you want more of them, it requires quite some scripting to aggregate the results, so a full-size scheduled script becomes more useful as you can fine-tune it to your needs.
Regarding the recursive next hop search, let me disagree with @Amm0 - while it is definitely more difficult to understand than a standalone netwatch, I consider this method far simpler than a script, because you just add N /32 routes to canary addresses and N default routes, all with the same distance but each using another canary address as gateway, and that’s it (if you set the scope of the routes to canaries and the target-scope of the default routes properly of course). You only need scripting if the DHCP server sometimes changes the gateway address. For Starlink in particular, you either get always 192.168.1.1 if using their bundled router or 100.64.0.1 if using a software or hardware bypass, so the recursive next-hop search comes script-free in your case. Yes, it does have that rigid failover time - 3 pings 10 seconds apart must get unresponded so that the canary was considered dead, but I haven’t had a single case of a false positive caused by Starlink short-term loss of connection when using it, even with a single canary. What I did have, though, was 8.8.8.8 unreachability via Starlink for minutes, at multiple geographically distant sites simultaneously whilst other internet destinations were reachable alright, so to me, the use of multiple canaries definitely makes sense.
There are cases where a failover after 30 seconds is way too late (some VPNs don’t tolerate such an outage); there, a full-size scripting is the only way, but then you can ping the canary multiple times per second and fail over within two seconds.
Regarding LTE - I’m not going to argue whether the LTE interface presents itself to the routing stack as being down if it loses the wireless connection - at least it doesn’t when the issue is account balance or alike, but the Ethernet on the SXT definitely doesn’t go down in such a case (well, unless you use scripting on the SXT to force it down but I don’t think it is a good idea to cut off access to device management whenever the LTE uplink stops working). There is indeed no point in tracking the state of the “WAN of last resort” from the point of view of failover (if all WANs are down, you’re offline, full stop), but for any redundancy setup, it makes a lot of sense to be informed about failures of the backup elements while the primary ones are alive - otherwise the unavailability of backup may remain unnoticed for months. So you may want to consider monitoring also the transparency of the LTE connection and using e-mail or Telegram to send notifications about the status changes.