Last months we are getting clients comments about erratic and random disconnects, movie freezing, pages partially loading.
But only only on clients that are PPPoE client 3rd party Wifi router. (So these 3rd party wifi router do the client auth. and are src-natting client traffic into our network and the internet.

Most clients have PPPoE client on the Mikrotik CPE.
Only clients with 3rd party CPE (Mimosa, eCambium or 60Ghz wAP clients) have PPPoE-client on 3rd party router (TP-link, Mimosa, Netis, etc.)

In the PPPoE server all Mikrotik clients are showing MTU of 1500 while all clients that authenticate with 3rd party devices show 1480 or 1492 as their 'actual MTU'
The comments and complaints come from (some) of these clients having 3rd party PPPoE-clients running.

So I dove into the MTU settings documentation online and on this forum but like always, too much info and now I'm lost in the forest....

All our gateway routers as well backhauls are basically Mikrotik gear.
PPPoE server setting (CCR1036 internet gateway router):
/ppp profile
dd address-list="" !bridge !bridge-horizon !bridge-path-cost !bridge-port-priority change-tcp-mss=yes dns-server=10.10.10.2 !idle-timeout !incoming-filter !insert-queue-before \
!interface-list local-address=192.168.200.1 name=defaultMTU1500noTCP_MSS on-down="" on-up="" only-one=default !outgoing-filter !parent-queue !queue-type !rate-limit !remote-address \
!session-timeout use-compression=default use-encryption=default use-mpls=default use-upnp=default !wins-server

/interface pppoe-server server
add authentication=pap,chap,mschap1,mschap2 default-profile=defaultMTU1500noTCP_MSS disabled=no interface=Bridge_PPPoE keepalive-timeout=30 max-mru=1500 max-mtu=1500 max-sessio
mrru=disabled one-session-per-host=yes pado-delay=0 service-name=internet

Mikrotik CPE typical:
/interface pppoe-client
add add-default-route=yes disabled=no interface=wlan1 keepalive-timeout=60 max-mru=1500 max-mtu=1500 name=pppoe-out1 \
service-name=internet use-peer-dns=yes user="Clientname+number"

#### so here we have both server and client working with mtu = 1500####

3rd party typical:
depending on the brand the Wifi router that is performing PPPoE auth has a mtu set of 1492 or 1498

###So here package leaving this router toward internet over our network, or towards client LAN are never bigger then this?#####
(Actually 99% of Mikrotik CPE clients that have PPPoE performed in the CPE also have 3rd party wifi router behind the CPE for the LAN. Again with 1492 or 1498 mtu. But no more PPPoE of course)

Connection between border gateway and client PPPoE device runs over wired and wireless MPLS network, in some parts Vlan's and off course some clients use VPN

What would be best advice for setting?

Need to set border gateway PPPoE server to lower mtu to allow fragmentation or disregard to take place in the border router for package from remote host destination client?

Or should I better work with two PPPoE server profiles. One service for Mikrotik devices, and one for 3rd party devices? Now I can set the optimum (or even bigger?) for Mikrotik network and smaller mtu for 3rd party?

Could somebody give me a 'mtu and PPPoE server for dummies' explanation so I can enhance the performance of our network and silence the complaints from some clients....?

...........
Could somebody give me a 'mtu and PPPoE server for dummies' explanation so I can enhance the performance of our network and silence the complaints from some clients....?

I would also like to ask how do you test MTU settings when running PPPoE Central Concentrator + MPLS + VPLS + VLAN's from CCR1009 connected to TP-link Jetstream L2 switch's and then
connected to several Netbox's, RB912, RB911, RB750, RB1100...

My PPPoE server uses
max-mtu=1480
max-mru=1480
mrru=1600

PPPoE Client
/interface pppoe-client monitor 0
mtu:1480
mru:1480

Just wondering is MRRU setting correct or should it be disabled?

So, w/PPPoE like other tunneling protocols we have to be aware of MTU along the path. We also have to think about how systems handle dissimilar MTU along the path of a packet.

To handle the issue around MTU along the path it's a fairly simple equation. With the default of 1480 for both in the PPPoE server you get 20 bytes of overhead for PPPoE across the path, with PPPoE dropping 8 bytes on it's own, if you add multi-link you lose 4 more bytes. If your environment runs default values of 1500 across the board you'll be at 1492 or 1488 for your real MTU at the PPPoE client interface. If you use multi-link with a MRU value of 1500 or greater you can send a single packet at greater than 1488 but it will create a small fragment to carry the remaining 12 bytes (meaning 2 packets total to carry 1 packet with a MTU between 1492 and 1501).

To handle dissimilar MTU along the path of a packet we use a mechanism called path MTU discovery or PMTUD. This relies on various mechanisms - most importantly ICMP. In IPv4 if a router receives a packet that is to large to emit down the path towards it's destination it can fragment it to fit into the link. It cannot fragment the packet if the DF bit is set though, this is common for TLS traffic that we encounter very commonly today. In this case the router will emit an ICMP Too Big message to the sender. It is not uncommon for the far side (out of your control) to drop this ICMP message. This is why TCP MSS is leveraged to help shape down the TCP related traffic to fit within the tunnel regardless of the MTU situation, of course this leaves the UDP flows broken (thankfully UDP is less common on the Internet by and large).

So to summarize, if your network's transport is operating at 1500 MTU your PPPoE server should be set to a MTU and MRU of 1492 with multi-link disabled. Additionally, you and your clients should not block ICMP Too Big messages to assist with Path MTU Discovery. If everything is set correctly and you are still having issues with PMTUD you can leverage TCP MSS to help alleviate the issues. If you do discover a far side issue with another service not respecting ICMP Too Big messages leaving your networks the best you can do is contact them and make them aware of their incorrect configuration. As a network admin - this is part of your life. The Internet is a shared and collaboratively operated network on a global scale and no one is perfect.

The MikroTik clients will have TCP MSS enabled by default which is likely helping your MikroTik clients stay connected and some of the 3rd party clients may not do that hence your issues becoming apparent.

An option is to enable your network to operate at a MTU that is greater than 1500 so your PPPoE services can be delivered at 1500 to the 3rd party end-points as well as the MikroTik clients. This has the added benefit of your network not being pointed to as a cause of issue for sites that have broken PMTUD that operate at 1500. Looking at the wiki you could determine what MTU values your equipment supports but you should be able to upgrade everything (if it's all MikroTik as you say) to at least an MTU of 1508 or 1512 (up to 1520). The next bump is at 2026. It's not unreasonable to have your network capable of moving baby jumbo frames. It gives you freedom to deploy overlay technology (PPPoE in this case, possible VPLS of EoIP in the future) without having to worry as much about the end MTU you're handing to your clients 3rd party routers.

Reference: https://wiki.mikrotik.com/wiki/Manual:M ... uterBoards