How can you load-balance between a pppoe gateway and a static gateway on the bridge?

If no route with a routing-mark matching the packet’s one is available, routing of that packet falls back to the default table “main” (which consists of all the routes without any routing-mark, the routing mark and routing table name are two names for the same thing, except that the name “main” can be used as an alias to “no routing-mark associated”).

That’s why, when you disable the routes with routing-marks, the firewall continues to assign connection marks and routing marks, but the routing marks are not used for routing so the mistakes in connection marking and routing marking do not spoil the process.

Can you post the complete output of /ip route export verbose? and /ip firewall export in the current state (the two routes with routing-mark disabled but the rest in place)? You can filter out the address-list from blocklist.de. Off topic, sending a reject to spam sources is a waste of network bandwidth, mere drop is better.

route.rsc (1.3 KB)
firewall.rsc (11.5 KB)

I don’t get it. I’ve told you that the purpose of the /ip route rule is to protect packets for LAN devices (in subnet 10.0.0.0/24) from being handled by routing-marked routes, and you have instead set it to handle packets towards 192.168.8.0/24 (which mostly do not exist).

If you modify that /ip route rule to deal with dst-address=10.0.0.0/24 and enable the two routes with routing-marks again, does it work or not?

As for NATing the traffic sent out via the two gateways as @alli recommends, don’t worry, the relevant /ip firewall nat rules are already there.

no does not work.

So two possibilities, either I have missed something else related to your unusual network setup, or there is some issue like in this topics.

So let’s start by disabling the special mangle rule for src-address=10.0.0.103, setting both the PCC rules to assign new-connection-mark=WAN1_conn and permitting the routes with routing-marks. After testing whether it works or not, set both PCC rules to assign new-connection-mark=WAN2_conn and try again. Report both results.

I tried again to insert 10.0.0.0/24 and the LAN seems to work
Schermata del 2018-07-07 15.25.12.png
then I try the additional instructions you gave me even if I didn’t understand exactly how to do it; in /ip mangle firewall I have to disable them all leaving the first two active?

sorry sindy,
I do not use anything ! I tried the /ip route rule above but it does not work .
Now I try with the pcc rules enabling one connection at a time

The idea is to find out more exactly which type of connections (those through PPPoE or those through LTE) causes trouble, as handling of each of the two types is slightly different.

So I wanted you to keep everything in the target configuration, except that you would systematically mark all connections to use PPPoE in the first step, and to use LTE in the second step. The easiest way to do that is to only set the new-connection-mark in both PCC rules to the same value (first to WAN1_conn, then to WAN2_conn). For the moment we can ignore connections initiated by packets coming from outside, as no such connections are permitted anyway, so the mangle rules in chain=input may remain unchanged.

Another important thing I forgot to mention is that fasttracking is incompatible with packet mangling. As the highest priority (lowest distance) default route in the default routing table uses pppoe-out1 (WAN1) as gateway, connections marked with WAN1_conn can be fasttracked; those using the LTE uplink (marked with WAN2_conn) must not be fasttracked. So first of all, add connection-mark=!WAN2_conn to the action=fasttrack-connection rule in /ip firewall filter. Only if that is not sufficient to make everything work, try the steps above (send everything through WAN1 and then everything through WAN2 using modified PCC rules).

The /ip route rule must stay in place all the time - assuming you try from a device on LAN.

I hope I have fully understood what needs to be done.
I activated the marked conections one at a time:
wan1 mangle.png
wan1 iproules .png
forWAN2:
wan2 iproules.png
wan2 mangle.png
does not work in both cases

Unfortunately you haven’t :frowning:

What I want to see is how it behaves when everything is configured the final way except the rules which contain the per-connection-classifier condition and the exceptional rule for src-address=10.0.0.103.
I want you to set the new-connection-mark in all these three rules to WAN1_conn. So all connections will be sent via pppoe-out1 but all the rest of the policy routing mechanism will be in use.

In the next step, you change the new-connection-mark in those three rules to WAN2_conn. So this time all connections will be sent via the LTE uplink.

I expect it to work in one setup (probably when everything is marked to use pppoe-out1) and not to work in the other one.

I don’t know how to do it; excuse me but I don’t have much experience :frowning: :frowning:

What is so complex about changing the new-connection-mark value to WAN1_conn in all rules which use that field?

then replacing the WAN2_conn field in all PCC rules; is it correct to do so?


/ip firewall mangle
add chain=input in-interface=pppoe-out1 connection-mark=no-mark action=mark-connection new-connection-mark=WAN1_conn
add chain=input in-interface=bridge src-address=!10.0.0.0/24 connection-mark=no-mark action=mark-connection new-connection-mark=WAN2_connWAN1_conn

add chain=output connection-mark=WAN1_conn action=mark-routing new-routing-mark=to_WAN1
add chain=output connection-mark=WAN2_connWAN1_conn action=mark-routing new-routing-mark=to_WAN2

add chain=prerouting dst-address-type=!local in-interface=bridge per-connection-classifier=both-addresses-and-ports:2/0 connection-mark=no-mark action=mark-connection new-connection-mark=WAN1_conn passthrough=yes
add chain=prerouting dst-address-type=!local in-interface=bridge per-connection-classifier=both-addresses-and-ports:2/1 connection-mark=no-mark action=mark-connection new-connection-mark=WAN2_connWAN1_conn passthrough=yes

add chain=prerouting connection-mark=WAN1_conn in-interface=bridge action=mark-routing new-routing-mark=to_WAN1
add chain=prerouting connection-mark=WAN2_connWAN1_conn in-interface=bridge action=mark-routing new-routing-mark=to_WAN2

/ip route
add dst-address=0.0.0.0/0 gateway=pppoe-out1 routing-mark=to_WAN1 check-gateway=ping
add dst-address=0.0.0.0/0 gateway=192.168.8.1 routing-mark=to_WAN2 check-gateway=ping

Yes! You’ve got it, this is what I had in mind (well, initially I wanted to keep the rules in chain=input untouched, but never mind as they aren’t used anyway until you want to access the system from WAN side).

Now, with WAN1_conn everywhere, does it work or not?

Then, next step, with WAN2_conn everywhere, same question :slight_smile:

Schermata del 2018-07-07 19.53.19.png
the two methods with wan1-conn and wan2-conn do not work

The blue text on the picture… did you always disable the two default routes (dst-address=0.0.0.0/0) without routing-mark when enabling those with routing-mark?

Because until now I was expecting the routing-marks to be assigned where they should not have been; if you have always disabled the default routes in table “main”, it could be that the routing marks were not assigned when they should have been.

So now please enable the routes with routing-mark but don’t disable those without, and try again with both “everything as WAN1_conn” an then “everything as WAN2_conn”.

And tell me how exactly you are testing, by attempting to open a web page from a browser on a laptop which has an address from 10.0.0.0/24 or some other way?

I kept the default routes active and tried WAN1_conn and WAN2_conn , but it still doesn’t work.
I have firefox on a computer with ubuntu and I have disabled web page caching, is connected in on the eth2 interface bridge

That’s incredible.

OK. Keep all the mark-connection rules set to WAN1_conn. Do /ip firewall mangle reset-counters-all and /ip firewall filter reset-counters-all Then make one attempt to open a web page. And then post the results of /ip firewall mangle print, /ip firewall mangle print stats, /ip firewall filter print, /ip firewall filter print stats.

ip firewall mangle print stats.txt (5.19 KB)
ip firewall mangle print.txt (4.05 KB)
ip firewall filter print stats.txt (3.13 KB)
ip firewall filter print.txt (1.61 KB)
we continue tomorrow or another day if you want, it was not my intention to engage you so much!
However, I can still do it, :laughing:

Well… did the reset-counters-all actually happen before you’ve printed the stats? There is so much traffic in the counters that it seems unlikely to me, plus when everything is connection-marked to WAN1_conn, there should be no hits on the “mark-routing to to_WAN2” rule and they are there.

But I start thinking whether it’s not the same issue which @Vagelis has encountered, as I wrote earlier, where upgrade to 6.42.5 was necessary.