EoIP slow high latency

RouterOS General
1

I have two mik’s connected by 3 or 4 routers and wireless radios,
call mikrp at the NOC and mikch out in the boonies. Wireless are Ubiquity Rockets running at over 100 megabits, and the routers inbetween are also miks. All
interfaces are 100 megabits, and all are clean of errors and drops.

Pings from mikrp to mikch shows about 10 to 20 ms but these pings do NOT
go over the EoIP.

PIngs to machines connected to mikch over the EoIP however are taking
200 to 400 msecs during ‘heavy usage’ time.

Usage is about 9 to 12 megabits download from mikrp to mikch which has
end users on it.

The results are dependable and repeatable every day during peak times. THe rest of
the link is empty, as there is no traffic on the link outside of the EoIP of significance.

THere is a second EoIP on the same link to yet another mik located after mikch,
and the ping times are also high there during peak times even though the
actual traffic on the EoIP is relatively low compared to the first EoIP.

This indicates that traffic on one link is enough to trouble both links, all the while
pings directly between miks remains low at all times.

mikrp -------- mikch ---------mikth
| |
customers customers
EoIP from mikrp to mikch
Eoip from mikrp to mikth

An ethernet bridge is created between mikrp and mikch and bridged so that
mikrp is a bridged extension of mikch.

The manual says the MTU of the EoIP and the bridge should be left at 1500 to
prevent fragmentation.

However the manual also says that the EoIP process adds 42 bytes to the
1500 bytes packet, 8 GRE, 20 IP, and 14 ethernet.

Since packets come into mikrp at 1500 like any other router, and mikrp
adds 42 bytes to the packet before putting it on the EoIP, it would seem
that fragmentation would have to happen at that point before putting the packet
on the EoIP.

However if the EoIP is set to 1458, then added 42 bytes makes it 1500 and
it fits on the EoIP without frag.

However since the packet came into mikrp at 1500 in the first place, it will clearly
need to be fragged to match the new 1458, so whether the MTU of the EoIP
is changed from 1500 to 1458 or not, does not change the fact that packets
will be fragged before being put on EoIP.

Is that right?

Further if the MTU of the EoIP is set to 1542, then the packets can go directly
onto the EoIP without fragging, but must be fragged before egressing that
router at 1500 on its way to the remote end of the EoIP tunnel.

Is that right?

Homer

2

Very similar problem and same question :wink:

3

we saw a similar problem with EoIP on RouterBoard hardware.
(However on decend x86 routers EoIP will run better)

Just go straight to MPLS/VPLS and/or VLANs.

This solved the issue for us.

EoIP IMO only is a quick-hack for emergency data-linking.

4

What kind of MT boxes do you use?

5

The EOIP problem happened across multiple different boxes, 564’s and
others.

We finally gave up the EOIP and worked a more direct line from
the end user to the internet.

The EOIP is still running from remote site to our local NOC, and
latencies are still in the 200 to 300 msecs, EVEN THOUGH the Eoip
is mostly empty.

The hardward links are not empty of course pushing 12 megs etc,
but their latencies are in the 20 msecs no matter how heavy they get.

So there is a definite problem with the EOIP that is not hardware
specific.

Homer Smith
CEO Lightlink Internet

6

Rb5xx might be to slow to get reasonable EOIP throughput.

7

Has there been any improvement on this? we are seeing the same issue using RB450G’s at two locations connecting over EOIP to a RB493AH. All the addresses for all the sites are held on the 493. Pinging the external interface (not over EOIP) we get 3-5ms but going over eoip we get 30-60ms.

8

Having the same problem here.

9

I have two RB2011 linked by a pair of ubnt Nanobridge. I have two EOIP tunnel on this link. I get a good steady 4~6ms response from both side of the tunnel from end devices.

10

Here I discovered the root cause of that issue with tunnels. UBNT airmax, when I turned the AIRMAX off, the ping latency time get the same inside the tunnel than outside.

11

weird, I’m using Airmax on my link and I’m not seeing this problem…

12

I upgraded to version 6.0 RC11 and I am still having the issues.

13

Did you check what is the CPU load? EoIP is very CPU hungry.

14

I know this is an older post, but we were struggling with the same issue of high latency across EoIP tunnels via wireless UBNT links. In a nutshell, we are using EoIP to tunnel legacy gear running in a flat layer 2 network across all our new routed links.

After a lot of screwing around we found the issue to be with a wireless RTS setting in the software running on our legacy WiFi gear. Default was RTS 2300 and we went in and set them all to RTS 256 - problem solved.

Scott

15

I know it has been 6 years on this, but I started the thread, so here is my wisdom over the years.

The MTU HAS to be set on the eoip interface to 1500 at both sides, as I remember the default at one time was
something other. Just fix it manually.

Second dont fragment has to be no.

Third there is some possibility that on high usage links over ubiquiti airmax, the frame aggregation, if on, will cause only huge
accumulated packets to be sent, thus increasing ping times, but not lowering bandwidth.

Fourth EOIP is simply slow, during afternoons with us normal pings are 6ms, and EOIP pings are 20ms. But during heavy airtime on the airmax radios, EOIP goes to 100 to 200 ms, while normal pings remain well normalish.

The advantage of EOIP is it allows a single subnet to be shared across two or more sites. That’s important for us because although the server farm has all the servers on the same subnet, (probably a really bad idea), if one machine goes down, I can bring it home and put it on our network at home with the same IP. No need to renumber.

Homer W Smith
CEO Lightlink

1 Like