First, I have a fairly large (geographically) network connected together over PTP wireless links. The links are routed with OSPF (no WDS/EOIP) and it works fairly well. For internet, all the sites route out the HQ internet connection which has a Sonicwall 3060 firewall. Two remote sites (HS & J) have internet connections and HQ has a second internet connection all for the purpose of site-to-site redundancy (internet traffic only routes out HQ’s primary internet connection) in the event a wireless link goes down.
Here is a simplified diagram for purposes of this conversation.
The wireless part had been working fine but I hadn’t configured redundancy. I opted for IPIP tunnels (J-to-HQ and HS-to-HQ) encrypted with IPSEC transport mode so I could run OSPF over them. That worked fine and I now have redundancy except for one problem… when traffic is routed over the IPIP tunnels I cannot use the internet at them. If I fail back to the wlan links it works fine. More specifically, I can ping out to the internet fine but HTTP web traffic doesn’t work. Sometimes it will pull a small part of the webpage but then it doesn’t load. All site to site traffic of all nature works fine including intranet HTTP traffic.
I feel like the 1480 MTU size is the problem (upsetting my sonicwall?) but I’m not sure. I tried all sorts of MTU settings changes to make it work but nothing helped.
Any ideas or suggestions on what to do or do differently? Switch to VPLS for the redundant links?
You should try to reduce the MTU size in the IPIP Tunnels.
MTU 1480 in the IPIP tunnel results in a 1500 Byte IP Packet (because of the duplicate IP Header).
This probably works fine with your internet connection. However, if you consider IPSEC ESP Overhead,
the resulting IP packet is larger than 1500 Bytes, and may be dropped anywhere in the transmission chain, rather than be fragmented.
From what I experienced with SonicWalls, they are very likely to silently drop stuff instead of fragmenting it or sending the corresponding
ICMP error back to the sender.
Try reducing the IPIP MTU to 1450 Bytes or less.
Also, add fitting Firewall Mangle Rules to adjust TCP MSS to 1410 (or, corresponding to whatever you set as IPIP MTU, even less),
so you don’t experience problems surfing the web.
Here are the overheads:
ESP in transport mode upper limit: 40 bytes
IPIP upper limit: 24 bytes
So you’d want an MTU of (1500 - 40 - 24) = 1436 bytes. Adjust this if you’re going to be doing stuff like NAT-T, where everything is mapped into UDP/4500 - if this number doesn’t work for you make sure you add up ALL the encapsulating protocols.
The TCP header is usually around 40 bytes. Adjust that to 64 just to be sure. Subtract that from the MTU, and set that as the MSS.
