We are an isp delivering wireless and fiber optic internet service. Some of our subscribers have complained that large files are stalling during their download and won't finish the download. We have looked at MTU, ports, firewall, nat/double-nat, and mangle rules, but don't have a firm bead on the actual root cause.
The subscribers are using a Mikrotik hAPax2 connected to an FTC11 back to a CRS328-4C-20S-4S+ fiber switch, which is connected to an RB5009 edge router that interfaces with our upstream provider on a 1Gig DIA circuit. All the firewall rules are on the edge router. There is NAT at the edge router, and at the home router. We are using unique VLANs for each subscriber with a /30 dhcp network for each VLAN, all networking is on the edge router with a DHCP client on the home router.
Some of the subscribers are using a 3rd party home wifi system such as Orbi or Eero. They have this issue, and so do people without the 3rd party gear. It seems to make no difference what gear is being used, except that when one of our subscribers used a TP link to augment their home network, and now his downloads of larger files work just fine.
Does this issue sound familiar to anyone out there who could take a crack at this?
Yeah — this pattern (small stuff works, large downloads stall/hang, different CPE brands mostly don’t matter, and one TP-Link “magically fixes it”) is classic Path-MTU / TCP MSS blackhole behavior.
In plain terms: somewhere between the customer and the internet, packets bigger than the real path MTU are getting dropped, and the mechanisms that should correct it (ICMP “Fragmentation needed” / PTB, or MSS clamping) aren’t working consistently. A lot of TP-Link gear clamps MSS by default, which “fixes” PMTU problems without you realizing it.