Weird LACP Issue

[xphat]

Hello all,


SO i’m trying to do a LACP Between a Mikrotik and the Switch Module of a Cisco 2851 Router to aggregate data between some ports.

Issue I am having here is that the Bonded Interface Comes Up However I am only getting Rx Traffic on One Interface but Tx Traffic on both interfaces.

SHO LACP NEIGHBOUR

Flags:  S - Device is requesting Slow LACPDUs
        F - Device is requesting Fast LACPDUs
        A - Device is in Active mode       P - Device is in Passive mode

Channel group 2 neighbors

Partner's information:

                  LACP port                        Oper    Port     Port
Port      Flags   Priority  Dev ID         Age     Key     Number   State
Fa1/0/44  SA      255       0060.e04a.6c03  20s    0x9     0x1      0x3D
Fa1/0/45  SA      255       0060.e04a.6c03  20s    0x9     0x2      0x3D

Bonding is setup like so:

0  R name="bond_Routed" mtu=1500 mac-address=00:60:E0:4A:6C:03 arp=enabled 
      slaves=ether5 - Bonding Connection To Internet,ether6 - Bonding Connection 
       To Internet 
      mode=802.3ad primary=none link-monitoring=mii arp-interval=100ms 
      arp-ip-targets="" mii-interval=100ms down-delay=0ms up-delay=0ms 
      lacp-rate=30secs transmit-hash-policy=layer-2-and-3

This LACP is a Layer 3 Port Channel, However I have another group 1 that is on a switchport between the same two devices that works as expected.


Anyone ever encountered this before?

[onnoossendrijver]

Can you tell me the layer 3 configuration of this part of the network?

If the Mikrotik only receives on 1 interface that’s because the Cisco sends only on 1 interface
The Cisco is the problem here.

What is the hash policy on the Cisco?

[xphat]

It is a /30 connection between the Cisco and the Router. Running OSPF for dynamic routing.

The transmit hash policy for the Cisco was the problem.

port-channel load-balance src-dst-ip


Fixed the Issue.


Thanks for the Direction.

[joegoldman]

this is one problem with proper 802.3ad from what i mostly see - its a per-flow kind of load balance means at best you might get a 70/30 balance, and a single stream always only 1 link worth. Mikrotiks own balance-rr gives proper bonding in my eyes, yet to get Mikrotik + Cisco doing good proper load sharing+bonding. I still use it though as it does split it a bit, and lets me lose a port/cable in failrue without losing the whole link.

[nz_monkey]

Yep problem is L2 hash on Cisco

[onnoossendrijver]

@joegoldman

this is one problem with proper 802.3ad from what i mostly see - its a per-flow kind of load balance means at best you might get a 70/30 balance, and a single stream always only 1 link worth. Mikrotiks own balance-rr gives proper bonding in my eyes, yet to get Mikrotik + Cisco doing good proper load sharing+bonding. I still use it though as it does split it a bit, and lets me lose a port/cable in failrue without losing the whole link.

Balance-RR can give severe problems when packets arrive out-of-order.

[nz_monkey]

@onnoossendrijver

@joegoldman

this is one problem with proper 802.3ad from what i mostly see - its a per-flow kind of load balance means at best you might get a 70/30 balance, and a single stream always only 1 link worth. Mikrotiks own balance-rr gives proper bonding in my eyes, yet to get Mikrotik + Cisco doing good proper load sharing+bonding. I still use it though as it does split it a bit, and lets me lose a port/cable in failrue without losing the whole link.

Balance-RR can give severe problems when packets arrive out-of-order.

Correct thats why the most compatible combo is to use 802.3ad + L3/L4 Hash on the Mikrotik side, and the same on the remote side.

If the remote side does not support L3/L4 hash, then use L3 hash, or L2 hash and it will all work happily.

We have this running perfectly:

Mikrotik to Cisco
Mikrotik to Juniper
Mikrotik to Extreme
Mikrotik to HP

[Splash]

I know this post is a bit old but would anyone be able to provide a working solution for creating a LAG between a Mikrotik and a Juniper router that actually works?

I’m really struggling to use a CCR1009-8G-1S-1S+ with its 1 x 10g and 1 x 1g fibre ports in a LAG. Traffic flows over the 1g, but not on the 10g. 10g port is forced to 1g (with a 1g SFP) and part of a bonded interface.
No matter what I do, the Juniper just does not want to bring the sfp+ with the 1g interface up.

Does this work with this model CCR?

Mikrotik sees it as up, but I dont think the LACP packets are getting to the Juniper…

show lacp interfaces ae1    
Aggregated interface: ae1
    LACP state:       Role   Exp   Def  Dist  Col  Syn  Aggr  Timeout  Activity
      ge-1/0/3       Actor    No    No   Yes  Yes  Yes   Yes     Slow    Active
      ge-1/0/3     Partner    No    No   Yes  Yes  Yes   Yes     Slow    Active
      ge-1/0/4       Actor    No   Yes    No   No   No   Yes     Slow    Active
      ge-1/0/4     Partner    No   Yes    No   No   No   Yes     Fast   Passive
    LACP protocol:        Receive State  Transmit State          Mux State 
      ge-1/0/3                  Current   Slow periodic Collecting distributing
      ge-1/0/4                Defaulted   Fast periodic           Detached

show lacp statistics interfaces ae1              
Aggregated interface: ae1
    LACP Statistics:       LACP Rx     LACP Tx   Unknown Rx   Illegal Rx 
      ge-1/0/3                  29          48            0            0
      ge-1/0/4                   0         711            0            0