GRE tunnel performance with RB4011 vs RB5009

I’ve got lots of Tik’s in service, probably 400-500 so familiar with the product lines, but, I have a hotel/hospitality group installation where we Tunnel the IP’s out to the site from our hosting facility. We acquired this customer from another consultant who disappeared off the face of the planet, however, he was buying the bandwidth from our colo and fortunately left the files with the customer and the passwords to the devices so we took over the customer.

With that said each location has an AT&T ABF connection one has a 300/300Mb fiber connection (rb4011) and the other has a 1Gb/1GB (RB5009). they connect on our side to a CCR1016 concentrator which handles the GRE tunnels and about 30 other customer tunnels back to us the CCR runs typically never higher than about 25% cpu. They are all running V6.x code, except my CCR which is running current V7. code

When i go onsite to these customers i am seeing about 700-740Mb/sec at the Gb customer and about 250-280Mb/sec on the 300Mb customer, for downloads to the customer. when i try to do upload testing the max upload speeds that i achieve is about 300Mb/sec on the 1GB and max 60Mb/sec on the 300Mb Customer. If i test Debian to Debian using iperf from behind the subscriber gateway, i get comparable results to the speedtest sites. If i test Tik to Tik with bandwidth test, i get full speed from tik to tik in both UDP and TCP so the circuit is in tact (so i think). The MTU on the AT&T service is (per MT auto detection) is 1476 on the connection and the GRE tunnels are all configured with a 1420 MTU as we do pass OSPF across the tunnel and 2 vlans.

It almost seems like the customer on the RB4011 seems to have some sort of hardware limitation on the tunneling (assuming it takes more horsepower to encode the data than decode it) and assuming that may be a similar issue on the RB5009. Is that a bad assumption? with other customers we have (all on ccr’s) they get symmetrical performance on there GRE tunnels. We have tried Wireguard and EOIP tunnels and the performance is worse.

questions:

1.) Should i be seeing symmetrical performance on these tunnels (they are not encrypted, no IPSEC or otherwise)?

2.) is there some setting that might be causing this?

3.) is this throughput issue perhaps generated by the hotspot and not the tunnels?

I have spare CCR’s so i may just have to go prove it to myself but was hoping someone knows the answer before i drive out there (they are about 100 miles away)

Any thoughts are welcome

Your mtu of 1476 on the at&t service seems highly dubious.

I believe the default MTU for at&t it is 1500. (But I could easily be wrong)

1476 matches the normal mtu of a gre tunnel. (Inside a normal 1500 mtu ethernet connection).

Then also 1420 seems odd, (looks maybe like an mtu for ipsec encrypted gre traffic)
Double check there are no ipsec policies in place? (possibly created manually separate to the gre interface)

But then 1420 with ipsec/gre is probably too big to fit in the 1476 at&t mtu?

My thoughts:

Set the at&t interface mtu to 1500

  • make sure you can ping places on the internet with 1500 byte no fragment packets.
  • (1472 (data) size if pinging from windows)
    Set the gre tunnel interface mtu to 1476
  • make sure you can ping devices inside other end of tunnel with 1476 byte packets
    ensure mss clamping is enabled on the gre tunnel interface.
    ensure there are no ipsec policies being applied.

If possible get it running in fast path.

Another thought, perhaps there is some queues configured.

I trialled a basic gre tunnel from a 4011 to a hap ax3, Edit: was incorrectly fast path (all fast track and about 2 meters of cable), and tested a bidirectional
btest over the gre link at 300M in both directions from my PC through the 4011 (gre) to ax3 (both gre and btest).

4011 had around 15% cpu. Ax3 was around 60%.

Throughput was 300M in both directions simultaneously. (I didn’t try for faster, but would clearly go faster)

Hi, we are facing the same issue with some RB4011 even with worse results. MTU of the connection is 1550 and GRE tunnel is 1500. Some ideas?