DoubleWAN Problems (maybe bc iam thinking to cheap) Starlink and 5GTether over EtherTether

Hey there,

first want to say: never seen a forum, where everybody seems so nice around here. No “youre stupid” or “let me google that for you “ Already happy to learn and be also of help .

”Standard Config” reffering Documentation :

“Standard Failover WAN Config refferring documenation":;

Column 1 Column 2 Column 3 Column 4
Model RB750Gr3
Revision r4
FirmwareType mt7621L
FactoryFirmware 6.46.8
CurrentFirmware 7.21
Upgrade Firmware 7.21
ether1 Starlink Gateway (v3 Router n Bypass) DHCP-Client WAN
ether2 Ethernet-USBC Active Adapter > Samsung S20 USB/Ethernettethering ON DHCP-Client WAN
ether3-5 bridge1 LAN
USBC→ Ether adapter https://www.amazon.de/dp/B0F13RJKJ3 just a ordinary USBC Thunderbolt 1000mbits converter with PD capability
/interface bridge add name=bridge1
/interface bridge port add interface=ether2 bridge=bridge1
/ip address add address=192.168.88.1/24 interface=bridge1

/ip dhcp-server/ setup
>>bridge1 as interface
>>dhcp address space 192.168.88.0/24
>>dhcp gateway 192.168.88.1
>>dhcp address pool 192.168.88.2-192.168.88.254
>>send dns ? YES
>>dns servers 192.168.88.1
>>releasetime 1800

/ip dhcp-client add disabled=no add-default-route=NO use-peer-dns=yes use-peer-ntp-yes interface=ether1
/ip dhcp-client add disabled=no add-default-route=NO use-peer-dns=yes use-peer-ntp-yes interface=ether2

/ip/route/ 
add dst-address=8.8.8.8 scope=10 gateway=100.64.0.1
add dst-address=8.8.4.4 scope=10 gateway=192.168.66.209
add distance=1 gateway=8.8.8.8 target-scope=11 check-gateway=ping
add distance=2 gateway=8.8.4.4 target-scope=11 check-gateway=ping


/ip firewall nat
  add chain=srcnat out-interface=ether1 action=masquerade
  add chain=srcnat out-interface=ether2 action=masquerade

/ip firewall filter
  add chain=input action=accept connection-state=established,related,untracked comment="accept established,related,untracked"
  add chain=input action=drop connection-state=invalid comment="drop invalid"
  add chain=input in-interface=ether1 action=accept protocol=icmp comment="accept ICMP"
  add chain=input in-interface=ether1 action=accept protocol=tcp port=8291 comment="allow Winbox";
  add chain=input in-interface=ether1 action=accept protocol=tcp port=22 comment="allow SSH";
  add chain=input in-interface=ether1 action=drop comment="block everything else";

/ip firewall filter
  add chain=forward action=fasttrack-connection connection-state=established,related \
    comment="fast-track for established,related";
  add chain=forward action=accept connection-state=established,related \
    comment="accept established,related";
  add chain=forward action=drop connection-state=invalid
  add chain=forward action=drop connection-state=new connection-nat-state=!dstnat \
    in-interface=ether1 comment="drop access to clients behind NAT from WAN"


/ip service set ssh port=2200
/ip service disable telnet,ftp,www,api
/ip dns set allow-remote-requests=yes

I have no special powers in regards to this ^^

Problems to solve:

Starlink on ether1 sometimes loses connection (bad placement, not to be placed elsewhere) and for those times, i want the ether2 port to jump right in, and take the traffic.

Specially for timesensitive things like GamePlay but also for “everyday” browsing, i hope, that i get no disconnects, or they stay small bc the failover takes the traffic, until the connection is established over the google DNS resolver again.

Momentarly i have the biggest problem, that the failover is not working. Second: the ether2 Port is having problems with renewal.

Somebody already tried something like this? or has advice besides, to invest into a 5G controller? (I looked around, they are to expensive for me at this moment)

Thanks a lot and a great weekend ahead chaps :smiley:

Haydar

Just for information:
If youre trying to have game during a failover scenario you will (usually) get disconnected in the game.

what does the “Router n Bypass” mean regarding starlink?

Also do you really need to check if the 5G is available?
Wouldn’t just a check on the Starlink be enough and then always fallback to the 5G?

Check that you also have both scopes and target scopes set properly (they may not show if they are default, but IMHO better make them explicit) and of course you only need to monitor first (main) connection, simply because you have not a third one to failover to.

if you prefer:
IF first connection is up -> nothing to do
IF first connection is down -> switch to second connection
THEN:
IF second connection is up -> nothing to do
IF second connection is down -> nothing to do

In case of need:
Simple recursive failover for bears of little brain

alternative (using netwatch instead of recursive):
Simpler Failover for two Gateways I found working - #19 by jaclaz

Thanks for your fast reply:

RouterNBypass:
Starlink GEN3 comes with the Dish (GEN3) and an Router/PoE “Thing” to supply the dish with power.

Normally this “Thing” can also do Router tasks (Like LAN DHCP, WLAN and so on) The Router of Starlink for that purpose is (friendly named) not really good for it.

Bypass Mode means, that the Router is behaving like a DHCP driven ISP Endpoint. You “just” plug into the Starlink “BypassRouter” port and you will get the Gateway, IP Adress and everything else (like also lease time ) from starlink.

Problem (maybe) is that the Gateway as also the IP Adress is always subject to change. In theorie every 5-15 Minutes.

GamePlay:
I hoped this relation can explain something easier than saying this:

I want to have no service interruption, if the dish fails for more than 2 seconds. Most of the everyday work is working on a CRM system i build wich is serversiderendered, wich also means, it does not really cares for network path based sessions.

Thank you so much for you help !

Just to be clear do you want the primary WAN to be starlink or 5G.
Understand you dont want the wans flapping back and forth, and you are saying that the starlink has lots of disconnects but not usually longer than 2 seconds, and thus only want it to be considered not available by the router if connectivity is down for more than 2 seconds??

As far as controlling the disconnects, it would be impossible and thus will lose game connectivity. The question really is, do you want to lose connectivity to the current session when Starlink comes back on line? That will of course happen with failover if you set up the router using standard recursive. If you want to ensure that any session using the 5G, continues regardless and only new sessions will move back to the Starlink, that can be done via Mangling on top of the standard recursive that jaclaz pointed too.

By the way, now that RoS include adjusting the ICMP ping interval and the number of checks before declaring the interface not available, there is less need for netwatch in basic scenarios.

@jaclaz Thanks a lot for your time :slight_smile:

Currently working thrue the sites you just mentioned :slight_smile:

@anav Primary WAN1 shall be the “not so reliable” starlink.

The main reason for concern, and the reason why “after 2” the switch shall “trigger”

Sometimes 70% of the time, the link is back in under 2 seconds. in 30% where it isnt, 50% of those downtimes are more than 5-120 seconds. The main reason i even bought the mikrotik is, that i want to automate exactly this

if the starlink fails for 2 seconds MAX, failover to 5G, before resuming Service on WAN1 (Starlink) test public dns resolver .

The “version3” configuration will be, that i want to establish a failover based on speed. Only bc the public dns resolver is ping true, does not mean, that the dish is supporting enough bandwith at the time.

This may sound crazy, and the closest non technical usecase maybe doomscrolling on social media. imagine, that IF the WAN1Starlink fails, after 2 seconds i want the reliable but really expensive 5G WAN used.

To be noted too: the failing starlink dish is not failing just once per 15 Minutes. Sometimes (depending on the count of sattelites in reach) it can happen more than 30-60 times in an hours. Sometimes time close, sometimes just 3-4 times per hour.

It would be better if you could post your whole configuration, instructions here:

Reading what you posted, these make little sense (to me)

You have a bridge with only one port in it (ether2) and - unless there are reasons for it, a bridge normally has two (or more) interfaces in it.

But later you treat ether2 like it was still "self-standing":

the gateway for the second connection is gateway=192.168.66.209 which doesn't belong to the static address you gave to the bridge, so likely the dhcp client on ether2 takes over and creates a dynamic route to it?

So, besides the full configuration, please post also the output of:
/ip route print
and of
/ip address print

@jaclaz Thank you much for your addittional help in this.

The postet script, was (with changes for the ether config, the exact config i used) For Forum beginners, we can not upload any data if this is our first post here.

Thanks for the hint guys!"

Normal “WAN1 active” WAN2 not in use

[admin@MikroTik] > /ip route print
Flags: D - DYNAMIC; I - INACTIVE, A - ACTIVE; c - CONNECT, s - STATIC
Columns: DST-ADDRESS, GATEWAY, ROUTING-TABLE, DISTANCE

DST-ADDRESS      GATEWAY     ROUTING-TABLE  DISTANCE

0  As 0.0.0.0/0        8.8.8.8     main                  1
1  Is 0.0.0.0/0        8.8.4.4     main                  2
2  As 8.8.8.8/32       100.64.0.1  main                  1
DAc 100.64.0.0/10    ether1      main                  0
DAc 192.168.66.0/24  ether2      main                  0
DAc 192.168.88.0/24  bridge1     main                  0
[admin@MikroTik] > /ip address print
Flags: D - DYNAMIC
Columns: ADDRESS, NETWORK, INTERFACE, VRF

ADDRESS            NETWORK       INTERFACE  VRF

0   192.168.88.1/24    192.168.88.0  bridge1    main
1 D 100.106.91.52/10   100.64.0.0    ether1     main
2 D 192.168.66.198/24  192.168.66.0  ether2     main
[admin@MikroTik] >

WAN2 needed, bc WAN1 faulty (first block directly after WAN1 looses connection, second 20 seconds after.

[admin@MikroTik] > /ip route print
Flags: D - DYNAMIC; I - INACTIVE, A - ACTIVE; c - CONNECT, s - STATIC
Columns: DST-ADDRESS, GATEWAY, ROUTING-TABLE, DISTANCE

DST-ADDRESS      GATEWAY     ROUTING-TABLE  DISTANCE

0  As 0.0.0.0/0        8.8.8.8     main                  1
1  Is 0.0.0.0/0        8.8.4.4     main                  2
2  As 8.8.8.8/32       100.64.0.1  main                  1
DAc 100.64.0.0/10    ether1      main                  0
DAc 192.168.66.0/24  ether2      main                  0
DAc 192.168.88.0/24  bridge1     main                  0
[admin@MikroTik] > /ip address print
Flags: D - DYNAMIC
Columns: ADDRESS, NETWORK, INTERFACE, VRF

ADDRESS            NETWORK       INTERFACE  VRF

0   192.168.88.1/24    192.168.88.0  bridge1    main
1 D 100.106.91.52/10   100.64.0.0    ether1     main
2 D 192.168.66.198/24  192.168.66.0  ether2     main
[admin@MikroTik] > /ip route print
Flags: D - DYNAMIC; I - INACTIVE, A - ACTIVE; c - CONNECT, s - STATIC
Columns: DST-ADDRESS, GATEWAY, ROUTING-TABLE, DISTANCE

DST-ADDRESS      GATEWAY     ROUTING-TABLE  DISTANCE

0  Is 0.0.0.0/0        8.8.8.8     main                  1
1  Is 0.0.0.0/0        8.8.4.4     main                  2
2  As 8.8.8.8/32       100.64.0.1  main                  1
DAc 100.64.0.0/10    ether1      main                  0
DAc 192.168.66.0/24  ether2      main                  0
DAc 192.168.88.0/24  bridge1     main                  0
[admin@MikroTik] > /ip address print
Flags: D - DYNAMIC
Columns: ADDRESS, NETWORK, INTERFACE, VRF

ADDRESS            NETWORK       INTERFACE  VRF

0   192.168.88.1/24    192.168.88.0  bridge1    main
1 D 100.106.91.52/10   100.64.0.0    ether1     main
2 D 192.168.66.198/24  192.168.66.0  ether2     main
[admin@MikroTik] >

full config textbased (now know, that i can just put the rsc in a texteditor, really great advice <3

Summary
2026-01-18 17:22:00 by RouterOS 7.21

software id = UKIT-MY8A



model = RB750Gr3

serial number = < mod edit: removed >

/interface bridge
add name=bridge1
/ip pool
add name=dhcp_pool0 ranges=192.168.88.2-192.168.88.254
/ip dhcp-server
add address-pool=dhcp_pool0 interface=bridge1 name=dhcp1
/interface bridge port
add bridge=bridge1 interface=ether3
add bridge=bridge1 interface=ether4
add bridge=bridge1 interface=ether5
/ip address
add address=192.168.88.1/24 interface=bridge1 network=192.168.88.0
/ip dhcp-client
add add-default-route=no interface=ether1
add add-default-route=no interface=ether2
/ip dhcp-server network
add address=192.168.88.0/24 dns-server=192.168.88.1 gateway=192.168.88.1
/ip dns
set allow-remote-requests=yes
/ip firewall filter
add action=accept chain=input comment="accept established,related,untracked" 
connection-state=established,related,untracked
add action=drop chain=input comment="drop invalid" connection-state=invalid
add action=accept chain=input comment="accept ICMP" in-interface=ether1 
protocol=icmp
add action=accept chain=input comment="allow Winbox" in-interface=ether1 
port=8291 protocol=tcp
add action=accept chain=input comment="allow SSH" in-interface=ether1 port=22 
protocol=tcp
add action=drop chain=input comment="block everything else" in-interface=
ether1
add action=drop chain=input comment="block everything else" in-interface=
ether1
add action=accept chain=forward comment="accept established,related" 
connection-state=established,related
add action=drop chain=forward connection-state=invalid
add action=drop chain=forward comment=
"drop access to clients behind NAT from WAN" connection-nat-state=!dstnat 
connection-state=new in-interface=ether1
/ip firewall nat
add action=masquerade chain=srcnat out-interface=ether1
add action=masquerade chain=srcnat out-interface=ether1
/ip route
add dst-address=8.8.8.8 gateway=100.64.0.1 scope=10
add check-gateway=ping distance=1 gateway=8.8.8.8 target-scope=11
add check-gateway=ping distance=2 gateway=8.8.4.4 target-scope=11
/ip service
set ftp disabled=yes
set telnet disabled=yes
set www disabled=yes
set ssh port=2200
set api disabled=yes
/system clock
set time-zone-name=Europe/Berli

TLDR current state

WAN1 Starlink is unreliable (ether1)
WAN2 is a Phone over ethernet for backup (ether2) But more expensive than the Starlink Connection

Current Setup, does not make the failover “alright”

Wishsetup/Goal

if WAN1 is failing for more than 2 seconds, failover to WAN2/ether2, go back if WAN1 is okay again

Ah, ok, ether2 is actually self-standing, not part of the bridge as initially posted, then everything is fine.

You could avoid a second masquerade rule in nat, by categorizing interfaces correctly, but the actually good thing would be to correct your firewall, basically right now it is only applying rule for ether1 (ether2 is NOT firewalled) and rules using interface lists are usually more readable.

Basically:

/interface list
add name=WAN
add name=LAN

/interface list member
add interface=ether1 list=WAN
add interface=ether2 list=WAN
add interface=bridge1 list=LAN

And then replace in-interface=ether1 and out-interface=ether1 with in-interface-list=WAN and out-interface-list=WAN in /ip firewall filter and /ip firewall nat.

The default recursive ping check of the gateway takes 10 seconds and checks every 10 seconds, and needs two timeouts to failover:

Gateway check can be extended by setting check-gateway parameter. Gateway reachability can be checked by sending ARP probes, or ICMP messages or by checking active BFD sessions. The router periodically (every 10 seconds) checks the gateway by sending either an ICMP echo request (ping ) or an ARP request (arp ). If no response from the gateway is received for 10 seconds, the request times out. After two timeouts gateway is considered unreachable. After receiving a reply from the gateway it is considered reachable and the timeout counter is reset.

so it is not suitable, but 2 seconds check seems to me a bit "aggressive" anyway.

With newish versions these settings can be changed:

but it has to be seen what happens with settings like:

Property Possible values?
check-gateway-ping-count (integer [0..65535], Default: 2) -->2
check-gateway-ping-interval (time [100ms..10m], Default: 10s) -->500ms
check-gateway-ping-timeout (time [1ms..1s], Default: 1s) -->500 ms

A netwatch based approach would IMHO be more flexible because the actions can be scripted, but with this relatively high polling rate and - possibly using the new enhanced ICMP/pinging - it is another rabbit hole to find the "right" parameters to NOT trigger false alarms while reliably failover in almost real time, a lot depends on how your connection (and your hex) behave in practice, see:

Suggest remove serial number from post, as a precautionary thing.

  • added a bit more security on the input chain, and its usual to stop using the default port for winbox, choose something random higher up and dont forget to write it down, 58921 for example.
    Only your admin pc should have access to the router for config purpose so an easy way is to make an address list of devices that belong to the admin.
  • allowing winbox on the WAN side is a big NO NO, it should only be accessible from the LAN SIDE.
    IF you want to gain access and since you dont really have a public IP address, you will need to use BTH Wireguard to access the router securely.
  • I wouldnt use SSH to gain access to the router but I suppose one cannot be too choosy, at least change the default port to something like 22566 (vice 22).
  • drop all other input chain traffic but ensure all new rules/lists are in place first.
    I set the ping router response to ping every 1 sec, to failover after the 3 ping so roughly at 3 secs.
    The ping timeout is 1sec so it will wait possibly till 4 sec before pulling the plug.
/interface bridge
add name=bridge1
/ip pool
add name=dhcp_pool0 ranges=192.168.88.2-192.168.88.254
/interface list
add list=WAN
add list=LAN
/ip dhcp-server
add address-pool=dhcp_pool0 interface=bridge1 name=dhcp1
/interface bridge port
add bridge=bridge1 interface=ether3
add bridge=bridge1 interface=ether4
add bridge=bridge1 interface=ether5
/ip neighbor discovery-settings
set discover-interface-list=LAN
/ip address
add address=192.168.88.1/24 interface=bridge1 network=192.168.88.0
/ip dhcp-client
add add-default-route=no interface=ether1
add add-default-route=no interface=ether2
/ip dhcp-server network
add address=192.168.88.0/24 dns-server=192.168.88.1 gateway=192.168.88.1
/interface list members
add interface=ether1 list=WAN
add interface=ether2 list=WAN
add interface=bridge1 list=LAN
add interface=WG-BTH list=LAN  { future rule }
/ip dns
set allow-remote-requests=yes
/firewall address-list   { ensure static addresses for dhcp leases }
add address=192.168.88.X list=AUTHORIZED comment="admin pC"
add address=192.168.88.Y list=AUTHORIZED comment="admin smartphone"
add address=192.168.88.Z list=AUTHORIZED comment="admin ipad"
/ip firewall filter
add action=accept chain=input connection-state=established,related,untracked
add action=drop chain=input connection-state=invalid
add action=accept chain=input protocol=icmp
add action=accept chain=input dst-address=127.0.0.1 
add action=accept chain=input  in-interface=bridge1 src-address-list=AUTHORIZED
comment="Admin access to config" 
add action=accept chain=input in-interface=WG-BTH src-address=RemoteDeviceWG-IP
{future rule}
add action=accept chain=input comment="allow SSH" in-interface-list=WAN port=22566 
protocol=tcp  { ensure secure SSH enabled in other settings }
add action=accept chain=input in-interface-list=LAN dst-port=53 protocol=udp
add action=accept chain=input in-interface-list=LAN dst-port=53 protocol=tcp
add action=drop chain=input comment="drop all else" {ensure above rules are in place first}
+++++++++++++++++++++++++++++++++++++++++++++++++++++++
add action=accept chain=forward connection-state=established,related
add action=drop chain=forward connection-state=invalid
add action=accept chain=forward in-interface-list=LAN out-interface-list=WAN 
comment="internet traffic"
add action=drop chain=forward comment="Drop all else"
/ip firewall nat
add action=masquerade chain=srcnat out-interface-list=WAN
/ip route
add check-gateway=ping dst-address=0.0.0.0/0  gateway=1.1.1.1 scope=30 \
target-scope=11
add dst-address=1.1.1.1/32  gateway=10.64.0.1 scope=11 target-scope=10
+++++++++++++++++++++++++++++++++++++++
add distance=2 dst-address=0.0.0.0/0 gateway=current actual LTE gateway comment=LTE
/ip service
set ftp disabled=yes
set telnet disabled=yes
set www disabled=yes
set ssh port=22566
set api disabled=yes
/routing settings
set check-gateway-ping-count=3 check-gateway-ping-interval=1s (ping timeout set 1sec )
/system clock
set time-zone-name=Europe/Berli
/tool mac-server
set allowed-interface-list=NONE
/tool mac-server mac-winbox
set allowed-interface-list=LAN

To set strong SSH (more secure you will need to go to the command line interface), aka select NEW TERMINAL in winbox. At the PROMPT Type

/ip ssh set strong-crypto=yes
/ip ssh set host-key-size=4096 host-key-type=ed25519
/ip ssh regenerate-host-key

Then go to system and reboot the router.

Remember, when Starlink comes back on line, the router will cut-off any sessions you have going on LTE, and you will have to start a new session, as you would when you lost Starlink in the first place.

The loss of the secondary ISP session/connection, due to the primary ISP coming back online, can be mitigated by mangle rules.

EDIT CONFIG WAS FAULTY- REMOVED

Done.

Thanks a lot for diving that deep. i do really appreciate that !

Well after you posted, most IMMEDIATLY i tried. and it exploded right in my face.

@jaclaz i tried your first suggestion with setting the checkping values as suggested.

The failover was nearly instantly (i monitored the TXRX values in the interface monitor.

i also observed, that all traffic in que, then tried to be handled in WAN2 but failed, i think due to the problem, that anav is explaining.

But wonderful to see, where i can dive even deeper into the documentation, and also thanks a lot “that you made my wish come true” nevertheless it is “good” to do so.

@anav Thank you so much for telling about the security issue at hand. i directly included that into my setup.

As i scolled thrue this forum and also thrue many other resources, iam finally better in explaining, what the real issue at hand is. (Thanks to ChatGPT, and iam sorry, for not beeing sorry to use it xD)

CHAT GPT
High-sensitivity quality triggers are required because the Starlink link rarely transitions to a hard “down” state, but instead exhibits short, recurring degradation events.
These events are characterized by brief packet-loss bursts and latency/jitter spikes caused by LEO satellite beam and path handovers.
Even sub-second degradation is sufficient to break or reset interactive, stateful protocols such as HTTPS and QUIC, while remaining invisible to traditional link-up/down monitoring.
Therefore, routing decisions must react to early quality degradation (loss, jitter, RTT variance) rather than waiting for complete link failure, in order to preserve user-perceived continuity for interactive traffic.

So exactly that what you are preconfiguring for me @anav. THANK YOU SO MUCH !

When i observe correctly, i learned this with both/all of you:

Mangeling is not “easy and done” needs fine tunining and is to be seen as a “not out of the box” its rather “try and error, and refine”

I will setup the router with your (anav) configuration, and will also log the hell out of my traffic (i already found the many different docu pages for that) and iam happy to share this here. (Goal would be to have a “user useful article” deployed with my name :smiley:

If i observe that correctly, for the “Quality” or for “UserExpirience” i will log

Ethernet link up / link down events on all WAN and LAN ports

  1. Ethernet RX/TX errors and drops on all interfaces
  2. Bridge port stability (LAN ports, no MAC flapping or topology changes)
  3. Continuous ping loss and RTT spikes on the Starlink WAN
    (multiple external targets, interface-bound pings)
  4. Netwatch up/down events for Starlink (signal only, time correlation)
  5. Default route and gateway changes (failover / recovery events)
  6. Firewall mangle rule counters (connection-mark / routing-mark activity)
  7. Connection tracking count during Starlink degradation
    (increased session churn as indicator of drops)
  8. WAN2 traffic usage (byte counters and short-term peaks)

Thansk so much for your support :slight_smile:

For logging purposes, i will use a high quality sd card, and of course watch CPU Usage, TEMP of device, and RAM Usage, to increase or decrease the logging . I will of course check the forum first. If some of you already have a forumpage in mind wich is useful, i hope that u may share it :smiley:

Best wishes for the beginning of this week
Best always

I would be surprized if my quick and dirty ICMP settings will address fully such a complex/fragile connectivity scenario, but hopefully it improves the experience. Jaclaz is correct in that netwatch can be more granular with more characteristics to address, but I have not seen anyone do it successfully ( its really really challenging and the documentation is not clear ) and they always revert back to simple...............

anavConfigANONym.rsc (3.6 KB)

I did not made out, what those “future config” things were, (tbh i typed everything one line by another into the terminal of WinBox bc of curiosity and the urge to see ^^)

add interface=WG-BTH list=LAN { future rule }
add action=accept chain=input in-interface=WG-BTH src-address=RemoteDeviceWG-IP
{future rule}

Do i understand that right, that this is the prepwork for wireguard?

iam just that happy that i wanted to share the current progress :smiley:

Will test now first and report back immediatly

who are “they” and what means “revert back to simple”? do they deactivate pieces in the RoS so everything you configured in that specific part is gone?

again thanks to both of you:D

NEVER had SUCH INCREDIBLE help in a VOLUNTARY forum really <3

Seems that "they" are "characteristics" an "they revert back" means "stay with simple numbers, do not try to be too smart as it always ends with simpler solutions".

@anav
Only seemingly off-topic, I tried to condense your mangle rules into an easier to be understood scheme, but I think I am missing something in the way it actually works.

Can you have a look at the attached and tell me what to correct/change?
EDIT: Removed, new version in a following post.

@mcaspen
As I expected, there is the need to monitor not only ping success (and response time) that can be considered a binary 0/1 or off/on state, but also other parameters, like jitter and RTT, that can only be checked by netwatch ICMP probe, and those are very difficult to tune/calibrate, you will need a lot of patience.

By "revert back to simple" anav is referring to the "other" possible probe in netwatch (the traditional one, that is actually called "simple" in Mikrotik settings), that is - nomen est omen - a simple binary check off/on.
And "they " are the people that try to use the ICMP probe, try to boldly tame its settings, and finally decide that the simple probe would do nicely.

Hi Jaclaz, You are right there was something missing on my rules.
The forward chain mangle reads should only apply to the first packet and thus also need connection-state=new

Looking at your chart......... keep in mind I am really only looking at what happens with a single connection, and that it starts out as a new connection with a first packet and subsequent packets.

  1. The first two rows would be blank/blank/blank/blank as there have been no connection marks assigned to traffic yet. The new connection would not do anything with the first two rows and thus I wouldn't have any entries.

  2. The next two prerouting rules, do apply as they stipulate a new connection, and we assign a routing mark to the traffic BUT not a connection mark. Routing has not actually occurred yet, but routing information has now been attached..(route is determined, but the packet has not yet left the out-interface so to speak) wan1-new rm-prefer-wan1 / NOTHING FOR WAN2 (wan1 is primary)

  3. The last two rules (forward), with connection-state=new also required for matching ( ensures only the initial packet matches the rule) assign a connection mark to the traffic heading towards wan1. The key here is now all the remaining packets gain this connection-mark. The packet goes out the door via ISP1.

  4. So looking at the next packet its no longer new, its established/related and thus the first two prerouting rules NOW apply, whereas ignored before. They reinforce/set the destination of the routing information based on the connection mark. The second set of rules are now ignored, no longer new.
    The last two rules are also ignored no longer new.

Thus, assuming WAN1 is up and running, all sessions are being conducted, and when WAN1 is found to be not available, the router fails to WAN2, and the above process begins again for new sessions except they will head to WAN2. WAN1 packets are interrupted and there is no response and eventually connections are removed.

The functionality is for the case of WAN1 coming back on line. What happens since Wan2 is not technically not available and has a perfectly functioning ISP interface and www connectivity. The mangling ensures that remaining packets dealing with WAN2, stay with WAN2 and continue to go through WAN2, despite WAN1 being the primary.