HELP WITH ROUTING 3 LAN TO DIFFERNET 3WAN IN SAME ROUTERBOARD

after greeting, please look to the fiqure first in attachment
i have one mikrotik router with 3 different isp i did the routing by route each lan to specific wan(pppoe)
LAN 1 TO WAN1
chain=prerouting action=mark-routing new-routing-mark=TOWAN1 passthrough=no src-address=192.168.1.0/24 in-interface= wifi1 log=no
log-prefix=“”
IN ROUTE…
dst-address=0.0.0.0/0 routing-table=TOWAN1 pref-src=“” gateway=pppoe1 distance=1 scope=30 target-scope=10 suppress-hw-offload=no
IN NAT
chain=srcnat action=masquerade src-address=192.168.1.0/24 out-interface=pppoe1 log=no log-prefix=“”
i did all these steps for each lan to specific wan pppoe with it mangel and route and nat
ITS WORK FINE BY 80% THE PROBLEM SOME APP AND WEBSITE AND SOME DOWNLOAD TASKS STOPS AND NOT WORKING EACH LAN HAS ITS UNIQE DNS AND CONFIGURATION, WHAT AM I MISSING AND WHAT THE SLOUTION TO DO IT IN RIGHT WAY LET SAY LIKE ONE ROUTER SPERATED TO 3 ROUTER

Untitled Diagram.jpg

When some TCP-based services have unspecified problems, the most frequent reason are issues related to Path MTU discovery if the WAN MTU is not 1500 bytes, which is your case here (PPPoE and even VLANs in use). So my first question is - have you tried to backup the configuration, remove all that “handmade VRF” setup you’ve implemented, and test those failing webs and apps with a basic setup consisting of only one of that “PPPoE over VLAN” WANs and only one LAN?

Once you do that and still encounter these problems, the next step would be to use mangle rules to force lower MSS using mangle rules in chain forward.

Why do you have to mangle to change MTU?

When I look at interfaces in winbox I see three settings, MTU, actual MTU and L2MTU ( the only one that seems changable is the first one )
Bridge settings also include MTU.

Because you don’t. The mangle rule doesn’t change the MTU - it works around an issue with path MTU discovery.

Well that just begs more questions!
Why do you want to circumvent MTU discovery?
What is MTU discovery?
WHY NOT just use MTU settings on interfaces??

https://en.wikipedia.org/wiki/Path_MTU_Discovery

So are you saying that MT devices cannot handle a standard ipv4 type protocol and is somehow corrupting the connection and thus we need to bypass MTs version of this protocol??

In other words, why does this not happen on lets say a cable connection or a FFTH fiber connection and also why not on all ppoe connections??
I will go further and state if its so bloody common, why doesnt MT make this a checkbox in the pppoe client settings… ignore MT MTU bastardized approach to MTU discovery.

Of course none of the above is true and its simply ISPs not following proper use of standards… ???

I am not fussed about the mechanics of how to put on a band aid or work-around (hated in software world) but the why, now thats interesting!

Where did I say that Mikrotik was doing something wrong? The mangle rule is used to work around the fact that the ICMP is blocked somewhere else on the path between the client and the server.

The MTU bottleneck is most likely the PPPoE connection - the L2 MTU of an Ethernet interface is typically 1514 bytes, but whereas a direct encapsulation of IP packets into Ethernet frames allows 1500-byte IP packets in this case, IP over PPP over Ethernet (PPPoE) takes some space for the PPP headers, so the IP MTU is lower in this case. And whereas many remote servers do get the ICMP “fragmentation needed” messages from the PPPoE server at ISP side, indicating the maximum packet size that can be delivered, some don’t. How would this mean that Mikrotik was causing this?

Well as per my post, I implied that its was not likely the MT but likely the sad state of ISP providers not providing a robust standard connection.
What you are saying is that its not even the ISP as its puts out appropriate information outbound of the MTU/ICMP requirements but other external sites/servers may not handle that information appropriately. So bypassing the MTU discovery on the MT helps how?? If the ISP is still putting out the MTU/ICMP requirements how does that help with external sites…

It’s not bypassing the MTU discovery, it’s substituting the MTU discovery. Instead of letting the PMTUD iteratively reduce the size of the TCP payload being sent by the server until it passes through, the mangle rule reduces the MSS value advertised by the client to the server. So the server sends small enough packets and PMTUD is not needed.

thanks for answer ,i tried to do one pppoe and one lan its work all fine with no problem i need to know how to force the in and out packet and traffic to go to some specific wan without any overlap and any missing in lan to wan traffic by the firewall and the mtu is not 1500 just in the first pppoe1 which is on ether interface and i dont think that is the problem

OK. So since you want to completely separate the routings between each PPPoE client and its corresponding LAN interface from each other, what about using VRF to assign the routing marks, rather than mangle rules?
Remove all routes from tables TOWAN2 and TOWAN3 and remove those tables from /routing/table, as these routing tables will be created dynamically in the next step:
/ip/vrf
add name=TOWAN2 interfaces=pppoe2,wifi2
add name=TOWAN3 interfaces=pppoe3,wifi3

This will create the routing tables TOWAN2 and TOWAN3, add routes to the subnets connected to wifi2 and wifi3 to them, and make sure that whatever enters the router via one of those interfaces will automatically use the appropriate routing table - no fallbacks to main, no missing match conditions in mangle rules. But you cannot use add-default-route=yes on the /interface/pppoe-client rows, as it would add the default routes via those interfaces to routing table main (hey Mikrotik, is it a bug or a feature?). So you have to add the default routes via PPPoE to their respective routing tables manually, with routing-table=TOWANx and gateway=pppoex@TOWANx.

soran,

Just see this is ROS7 !!!
there is change in the route target-scope parameter to be set (look up the post on ROS7 recursive routing)

As result, Target scope must be same or bigger than scope (not the case in your setup, which has worked in ROS6 I suppose).
I solved my routes which had similar issue than yours by setting scope and target-scope to the same…

i will try this thank you for the answer you are great helper person

i will try it thanks alot

this worked so fine …no problem any more thank you so much