Hi

We run an ISP, we have a core router with two 1 gig connections passing through to an edge router. From a ccr1036 to a ccr1072.
We are using balance-rr for bonding - to get "2gbps" throughput.

Strange issue that was brought up by a client getting slow speeds to their sharepoint server overseas, they are only getting 1-2Mbps on download/upload even though they are on a 100Mbps connection. However the client can download e.g a 100Mb bin file from e.g linode or any other international server at 50-100Mbps i.e we have enough international capacity.

So we tagged a vlan through the core to the edge and setup IP's instead of using pppoe - same slow speeds.
However if we tagged the vlan and bypassed the bonding and went over a spare interface between the two routers speeds were immediately 500-1000% better, i.e at least 10-20Mbps to the sharepoint server.

Another issue was overseas speedtest.net test results, full download speeds over the bonding interface but only +-5Mbps upload speed. We used the vlan and by bypassing the bonding and we could get full download speeds and much better upload speeds.

Could this be a TCP retransmit issue on the bonding interface? Especially affecting higher latency connections? If we use the btest to a public btest server in the states - there is no issue even though the latency is +-220ms or so - however this could use multiple threads.

You think we should change to LACP, i.e 802.3ad instead of balance-rr?

Thanks

Good topic I had also same problem.. i test 802.3ad for bonding two interface between ccr1036 and ccr1072 but it was not worked.. 802.3ad works betwen mikrotik and cisco. But betwen two mikrotik i could not get succsess...

It is very strange, only affecting some forms of traffic. I can only presume due to packets coming over different interfaces and being renumbered or such causing delays and TCP re-transmissions. Which would affect higher latency traffic i.e international.

Local traffic there is no issue, probably due to the low latency of 1-20ms.

It is very strange, only affecting some forms of traffic. I can only presume due to packets coming over different interfaces and being renumbered or such causing delays and TCP re-transmissions. Which would affect higher latency traffic i.e international.

Local traffic there is no issue, probably due to the low latency of 1-20ms.

The problem is balance-rr itself. If TCP packets are coming out of order, congestion control system will kick in by retransmiting segments.
If you have the cpu power 802.3ad would do a better job in your case.

Just letting everyone know, switching to 802.3ad i.e lacp, solved our issue. We are now getting full speeds to overseas on higher latency paths.
Super happy. And ps - this is between a ccr1016 and a ccr1072 - so Mikrotik to Mikrotik using layer2 + 3 hashing.

Just letting everyone know, switching to 802.3ad i.e lacp, solved our issue. We are now getting full speeds to overseas on higher latency paths.
Super happy. And ps - this is between a ccr1016 and a ccr1072 - so Mikrotik to Mikrotik using layer2 + 3 hashing.

Hi Paulct,
I test 802.3ad before but cant get success betwen CCR1072 and CCR1036. Can you share your settings I want to re try 802.3ad again.

Thanks.

Hi Paulct,
I test 802.3ad before but cant get success betwen CCR1072 and CCR1036. Can you share your settings I want to re try 802.3ad again.
Thanks.

Sure, simple config. I am only posting the config from the one router, as the other is exactly the same. The obliviously a /30 on the bonding interface, OSPF etc. Nothing else. Working really well.

> interface bonding print
Flags: X - disabled, R - running
0 R name="bonding-core" mtu=1500 mac-address=6C:3B:6B:E2:44:67 arp=enabled arp-timeout=auto
slaves=sfp5-procera-downlink,sfp-sfpplus7-procera2 mode=802.3ad primary=none link-monitoring=mii
arp-interval=100ms arp-ip-targets="" mii-interval=100ms down-delay=0ms up-delay=0ms lacp-rate=30secs
transmit-hash-policy=layer-2-and-3 min-links=0