My company and our upstream internet provider are setting up LACP link aggregation, on their end it’s Maipu MyPower 3400 and on our is Mikrotik CCR1036.
Thing is, we don’t have acess to Maipu and we don’t have anyone skilled with Maipu devices. On Mikrotik side, settings are pretty straightforward, create bonding with 2 slave interfaces, select 802.3ad mode, MII monitoring mode and that’s it.
But, as you can see, our tx is shared by 2 bonded interfaces (ether11 and ether12), but our rx on ether12 is zero (or 1064bps at this moment) and ether11 is receiving everything.
When I remove the cable from ether11, ether12 takes over everything, but as soon as I plug it in, it returns to previous state.
This is setting on interfaces>bonding
I don’t know much about Maipu, but I requested setup from upstream provider, so if anyone can tell if there is anything wrong with setup it would be nice
maipu-device#sh link-agg group 2
Link Aggregation 2
Mode: LACP Description: MikroTik-bond
Load balance method: src-mac
Number of ports in total: 2
Number of ports attached: 2
Root port: port 0/25
port 0/25: ATTACHED
port 0/27: ATTACHED
port 0/25 link aggregation 2 mode: Active
selected: YES
attached: YES
lacp_enabled: Lacp_Enabled_Full_Duplex
actor information:
32768.00017A584948.32768.26=0x0702
Active LongTimeout Aggregatable IN_SYNC Collecting Distributing Received Unexpired
partner information:
65535.4C5E0C46B816.255.1=0x0011
Active LongTimeout Aggregatable IN_SYNC Collecting Distributing Received Unexpired
state machine information:
receive=CURRENT, periodic=SLOW, mux=COLLECTING_DISTRIBUTING
port 0/27 link aggregation 2 mode: Active
selected: YES
attached: YES
lacp_enabled: Lacp_Enabled_Full_Duplex
actor information:
32768.00017A584948.32768.28=0x0702
Active LongTimeout Aggregatable IN_SYNC Collecting Distributing Received Unexpired
partner information:
65535.4C5E0C46B816.255.2=0x0011
Active LongTimeout Aggregatable IN_SYNC Collecting Distributing Received Unexpired
state machine information:
receive=CURRENT, periodic=SLOW, mux=COLLECTING_DISTRIBUTING
maipu-device#sh run link-agg 2
Building Configuration...
link-aggregation 2
port-type nni
description MikroTik-bond
switchport access vlan 10
exit
maipu-device#sh port 0/25
port 0/25 configuration information
Description :
Status
: Enabled
Link
: Up
Set Speed
: Auto
Act Speed
: 1000
Set Duplex
: Auto
Act Duplex
: Full
Set Flow Control : Off
Act Flow Control : OffMdix
: Normal
Mtu
: 1824
Port mode
: LAN
Port ability : 1000M FD
Link Delay
: No Delay
Storm Control : Unicast Disabled
Storm Control : Broadcast Disabled
Storm Control : Multicast Disabled
Storm Action : None
Pvid
: 10
Set Medium
: Fiber
Act Medium
: Fiber
Mac Address : 0001.7a58.4948
maipu-device#sh port 0/27
port 0/27 configuration information
Description :
Status
: Enabled
Link
: Up
Set Speed
: Auto
Act Speed
: 1000
Set Duplex
: Auto
Act Duplex
: Full
Set Flow Control : Off
Act Flow Control : Off
Mdix
: Normal
Mtu
: 1824
Port mode
: LAN
Port ability : 1000M FD
Link Delay
: No Delay
Storm Control : Unicast Disabled
Storm Control : Broadcast Disabled
Storm Control : Multicast Disabled
Storm Action : None
Pvid
: 10
Set Medium
: Fiber
Act Medium
: Fiber
Mac Address : 0001.7a58.4948
maipu-device#sh link-agg 2 statist
link-aggregation 2 statistics information:
RxOctets
: 289059206042
RxUcastPkts
: 983607045
RxMulticastPkts
: 2827
RxBroadcastPkts
: 470
TxOctets
: 1096892441825
TxUcastPkts
: 1192478926
TxMulticastPkts
: 1932
TxBroadcastPkts
: 430
TotalOctets
: 1385951647867
TotalPkts
: 2176091626
TotalUcastPkts
: 2176085971
TotalBroadcastPkts
: 900
TotalMulticastPkts
: 4759
TotalCRCErrors
: 0
TotalSymbolErrors
: 0
TotalAlignmentErrors : 0TotalUndersizePkts
: 0
TotalOversizePkts
: 0
TotalFragments
: 0
TotalJabbers
: 0
TotalCollisions
: 0
TotalPkts64Octets
: 326196333
TotalPkts65to127Octets : 824437807
TotalPkts128to255Octets : 112071022
TotalPkts256to511Octets : 32976935
TotalPkts512to1023Octets : 35261920
TotalPkts1024to1518Octets: 845147609
input rate 111866496 bits/sec, 50769 packets/sec
output rate 474421822 bits/sec, 63603 packets/sec
maipu-device#sh port 0/25 statist
port 0/25 statistics information:
RxOctets
: 152540900705
RxUcastPkts
: 505382589
RxMulticastPkts
: 1416
RxBroadcastPkts
: 0
TxOctets
: 1097845190413
TxUcastPkts
: 1193535646
TxMulticastPkts
: 982
TxBroadcastPkts
: 45
TotalOctets
: 1250386091118
TotalPkts
: 1698920674
TotalUcastPkts
: 1698918235
TotalBroadcastPkts
: 45
TotalMulticastPkts
: 2398
TotalCRCErrors
: 0
TotalSymbolErrors
: 0
TotalAlignmentErrors : 0
TotalUndersizePkts
: 0
TotalOversizePkts
: 0
TotalFragments
: 0
TotalJabbers
: 0
TotalCollisions
: 0
TotalPkts64Octets
: 206915223
TotalPkts65to127Octets : 571901955
TotalPkts128to255Octets : 87919848
TotalPkts256to511Octets : 25150568
TotalPkts512to1023Octets : 27907207
TotalPkts1024to1518Octets: 779125872
input rate 49319555 bits/sec, 25569 packets/sec
output rate 473903746 bits/sec, 63653 packets/sec
maipu-device#sh port 0/27 statist
port 0/27 statistics information:
RxOctets
: 136942230925
RxUcastPkts
: 479727233
RxMulticastPkts
: 1414
RxBroadcastPkts
: 471
TxOctets
: 355791311
TxUcastPkts
: 371560
TxMulticastPkts
: 954
TxBroadcastPkts
: 385TotalOctets
: 137298022236
TotalPkts
: 480102017
TotalUcastPkts
: 480098793
TotalBroadcastPkts
: 856
TotalMulticastPkts
: 2368
TotalCRCErrors
: 0
TotalSymbolErrors
: 0
TotalAlignmentErrors : 0
TotalUndersizePkts
: 0
TotalOversizePkts
: 0
TotalFragments
: 0
TotalJabbers
: 0
TotalCollisions
: 0
TotalPkts64Octets
: 119684799
TotalPkts65to127Octets : 253767939
TotalPkts128to255Octets : 24321002
TotalPkts256to511Octets : 7866612
TotalPkts512to1023Octets : 7396803
TotalPkts1024to1518Octets: 67064862
input rate 62695735 bits/sec, 25295 packets/sec
output rate 867 bits/sec, 0 packets/sec
I believe this is the problem.
802.3ad is not per packet balancing method. Since source MAC address is always the same all traffic goes over one link.
In RouterOS you can specify transmit hash policy layer2 and layer3 to create hash of mac and IP combination. It should be possible to change balancing method on Maipu, too.
Thanks. After doing some reading last night I came to the same conclusion. On Maipu there is no layer 2 and 3, there is only balancing by IP (L3) or by MAC (l2) if I understood colleague from upstream ISP. We will try changing balancing mode to ip address.
From working with many vendors switches and routers the terminology is quite different between vendors, also they have different default hashes.
Generally:
L3 = L2/L3 hash
L4 = L3/L4 hash
HP Procurve defaults to a L2 hash
Juniper EX and QFX defaults are a L3/L4 hash
Cisco 26xx/37xx are a L2 by default
Also some vendors will say their L3/L4 hash is proprietary, from experience this is not the case. We have Extreme ↔ Juniper, Extreme ↔ Mikrotik, Juniper ↔ Mikrotik, Alcatel-Lucent 7210/7750 ↔ Mikrotik all working using L3/L4 hash in both directions.
Also remember the hash is egress only. So even if only one end supports a particular hash it generally works.