Hello, I have created some GRE tunnels btw 3 routers:
uk1 → mad1 → ali1
uk1 GRE:
[login@uk1] > interface gre print
Flags: X - disabled, R - running
0 R name="mad1" mtu=auto actual-mtu=1476 local-address=IP.uk1 remote-address=IP.mad1 dscp=inherit clamp-tcp-mss=yes dont-fragment=no
[login@uk1] >
mad1 GRE:
[login@mad1] > interface gre print
Flags: X - disabled, R - running
0 R name="ali1" mtu=auto actual-mtu=1476 local-address=IP.mad1 remote-address=IP.ali1 dscp=inherit clamp-tcp-mss=yes dont-fragment=no
1 R name="uk1" mtu=auto actual-mtu=1476 local-address=IP.mad1 remote-address=IP.uk1 dscp=inherit clamp-tcp-mss=yes dont-fragment=no
[login@mad1] >
ali1 GRE:
[login@ali1] > interface gre print
Flags: X - disabled, R - running
0 R name="mad1" mtu=auto actual-mtu=1476 local-address=IP.ali1 remote-address=IP.mad1 dscp=inherit clamp-tcp-mss=yes dont-fragment=no
[login@ali1-router] >
The issue that I have is receiving e-mails in the routed IPs via the GRE.
In the Exim log I get the error:
SMTP data timeout (message abandoned) on connection from sender.domain.com
Looking on exim doc:
It means that there was a timeout while Exim was reading the contents of a message on an incoming SMTP connection. That is, it had successfully accepted a MAIL command, one or more RCPT commands, and a DATA command, and was in the process of reading the data itself. The length of timeout is controlled by the smtp_receive_timeout option.
If you get this error regularly, the cause may be incorrect handling of large packets by a router or firewall. The maximum size of a packet is restricted on some links; routers should split packets that are larger. There is a feature called “path MTU discovery” that enables a sender to discover the maximum packet size over an entire path (multiple Internet links). This can be broken by misconfigured firewalls and routers. There is a good explanation at > http://www.netheaven.com/pmtu.html> . Reducing the MTU on your local network can sometimes work round this problem. See Q0017 (3) for further discussion.
Broken path MTU discovery is one of the many woes of networking caused by people just discarding all ICMP
(I don’t mean you - I mean “people”)
Try making a mangle rule to clamp the mss of the exim host at whichever router is closest to it. Clamp it silly crazy low like 1200 and see if that makes any difference for you. Move forward from there.
If pmtu discovery is broken in general, I would expect the mangle rule to fail also.
The only issue with 1500 MTU in GRE is that your routers will now be performing packet fragmentation / reassembly, which can be anything from a complete non-issue to a show-stopping disaster. It depends on how much bandwidth goes through the routers, how much fragmentation happens (100Mbps of VoIP would not matter, because it is already small packets, for instance), and how much horsepower your routers have.
Just watch system load and if the routers don’t get overloaded, then you should be okay.
With MTU 1500 on the GRE tunnels, the issue that we detect is that wget downloads from servers connected to mad1 or ali1 and with a IP routed via the GRE (a protected IP) never finish… the download start, but not finish.
Also, if I change the MTU to 1476 (default), the download is Ok, but I have problems with the e-mails (main issue)
I sent a long PM about this - but for the sake of the thread, try disabling the ICMP filter, and post here whether that fixes the issue. (even if you stop blocking your own outgoing “icmp fragmentation required” messages, someone else might also be filtering them)
This icmp message is critical in the functioning of path mtu discovery.
. . . or they can DDoS any host behind the core router, and the link will be flooded anyway…
. . . or they could just send it to random IPs all over your ranges because they don’t care - they just want to flood everything. Plus you can’t blackhole a victim IP in some tier1’s backbone if your entire CIDR block is the victim IP…
If you want to harden the core router against CPU load from processing so much traffic - Put a rate-limit on input chain for ICMP Echo Request ~ 5 per second, burst = 10. This is far below DDoS levels, far above throwing away all ICMP of all types just to block pings. Heck, while you’re at it, make an IP address list of managment servers (NOC IP addresses, etc) and put a rate limit input for everything not in that list, and not routing protocols from your own routers and valid ebgp neighbors…
A friend and I were having a conversation just last night on this very topic.
Everyone knows where CIA headquarters is.
It’s on Google Earth
Try to go break into it.
