Community discussions

MikroTik App
 
User avatar
Ibersystems
Forum Guru
Forum Guru
Topic Author
Posts: 1686
Joined: Wed Apr 12, 2006 12:29 am
Location: Cabrils, Barcelona - Spain
Contact:

Redundancy with 2 bridge and LACP switchs

Tue Feb 19, 2008 2:12 pm

Hi all,

I need to make a double wireless link with 4xRB333.

My costumer have Allied Telesyn switch LACP capable and we would use this to offer redundancy.

If I configure the 2 links with WDS and interfaces in a bridge (link in transparent mode) this should work? LACP packets will pass over the link without problems?

I have to make some changes to the links configuration?


Thanks,
Martín.
 
User avatar
nz_monkey
Forum Guru
Forum Guru
Posts: 2096
Joined: Mon Jan 14, 2008 1:53 pm
Location: Over the Rainbow
Contact:

Re: Redundancy with 2 bridge and LACP switchs

Tue Feb 19, 2008 10:16 pm

Hi Martin,

I am pretty sure this will not work as the Linux bridging protocol interferes with the packet stream. A good example is that we tried to set up redundant links between a Cisco 3750 and a Cisco 3560 using a Proxim/Terabeam GigaLink and a Mikrotik Link using 2xRB333's and MSTP. This did not work as the Mikrotik links were not passing the spanning tree packets.

Hope this helps.
 
User avatar
Ibersystems
Forum Guru
Forum Guru
Topic Author
Posts: 1686
Joined: Wed Apr 12, 2006 12:29 am
Location: Cabrils, Barcelona - Spain
Contact:

Re: Redundancy with 2 bridge and LACP switchs

Wed Feb 20, 2008 1:30 am

Hi and thanks Nz!


If LACP will not work, then, how can I make this 2 links with redundancy? I make an identical scenario with 2 Alvarion Breezenet B100 links.

There is any way to make it possible with mikrotik?


Thanks!
Martín.
 
User avatar
Ibersystems
Forum Guru
Forum Guru
Topic Author
Posts: 1686
Joined: Wed Apr 12, 2006 12:29 am
Location: Cabrils, Barcelona - Spain
Contact:

Re: Redundancy with 2 bridge and LACP switchs

Mon Feb 25, 2008 4:03 pm

Hi all!


It can work with EoIP tunnels?


Thanks,
Martín.
 
User avatar
nz_monkey
Forum Guru
Forum Guru
Posts: 2096
Joined: Mon Jan 14, 2008 1:53 pm
Location: Over the Rainbow
Contact:

Re: Redundancy with 2 bridge and LACP switchs

Mon Mar 03, 2008 1:29 am

Hi Martin,

Not too sure on that one, maybe Uldis can help ?

MT always seem to go quiet when issues regarding MSTP, LACP and the like come up ;)

It would be nice if they could integrate the ZebOS stack into RouterOS, it would fix a number of issues that currently exist and I wouldnt mind paying a bit more for MSTP and LACP support.
 
User avatar
Ibersystems
Forum Guru
Forum Guru
Topic Author
Posts: 1686
Joined: Wed Apr 12, 2006 12:29 am
Location: Cabrils, Barcelona - Spain
Contact:

Re: Redundancy with 2 bridge and LACP switchs

Mon Mar 03, 2008 1:56 am

Hi! 3 days trying different solutions, but anyone seams to work..

Uldis, there is any solution to make my costumer LAN working with redundancy? I'm in a trouble now..


Thanks,
Martín
 
User avatar
nz_monkey
Forum Guru
Forum Guru
Posts: 2096
Joined: Mon Jan 14, 2008 1:53 pm
Location: Over the Rainbow
Contact:

Re: Redundancy with 2 bridge and LACP switchs

Mon Mar 03, 2008 5:41 am

Hi Martin,

What you can do is run RSTP accross the bridge, this is ugly but it will work.
 
User avatar
Ibersystems
Forum Guru
Forum Guru
Topic Author
Posts: 1686
Joined: Wed Apr 12, 2006 12:29 am
Location: Cabrils, Barcelona - Spain
Contact:

Re: Redundancy with 2 bridge and LACP switchs

Mon Mar 03, 2008 10:58 am

Hi nz! but there is a problem with this..

If I have 2 mikrotik in each side, connected to my switch then the RJ45 switch connector allways will work. If I lose my wireless link these connectors will be working.. I think I can lose data with this.


Martín.
 
User avatar
nz_monkey
Forum Guru
Forum Guru
Posts: 2096
Joined: Mon Jan 14, 2008 1:53 pm
Location: Over the Rainbow
Contact:

Re: Redundancy with 2 bridge and LACP switchs

Wed Mar 05, 2008 4:02 am

Hi Martin,

No it will use one or the other Mikrotik link. I have this operating at a few sites with Cisco 3750 --> Cisco 3560 links.


Regards,



Andrew
 
User avatar
Ibersystems
Forum Guru
Forum Guru
Topic Author
Posts: 1686
Joined: Wed Apr 12, 2006 12:29 am
Location: Cabrils, Barcelona - Spain
Contact:

Re: Redundancy with 2 bridge and LACP switchs

Thu Mar 06, 2008 6:20 pm

Hi all,

my trouble with LACP and all relationated with my double link continues very bad.

I tryed using RSTP in both sides (RB333) and in switchs and the link is redundant but only one link works at same time and the other link continues unused.
I tryed using EoIP tunnels in the 2 links and all works same than in last case. LACP packets can't pass over the link and switch ports aren't syncronized.

Can anyone help me with this? I think Uldis isn't reading this post : /

I need to make this link working at about 100 Mbps UDP (50 of one RB333 link and 50 with the other link)

My costumer will crush my head with the 4 RB333 : O!


Thanks,
Martín.
 
changeip
Forum Guru
Forum Guru
Posts: 3830
Joined: Fri May 28, 2004 5:22 pm

Re: Redundancy with 2 bridge and LACP switchs

Thu Mar 06, 2008 6:33 pm

have you already configured per the bonding requirements on RouterOS? It also looks to use LACP to handle things.

http://www.mikrotik.com/testdocs/ros/2. ... ontent.php

Sam
 
User avatar
Ibersystems
Forum Guru
Forum Guru
Topic Author
Posts: 1686
Joined: Wed Apr 12, 2006 12:29 am
Location: Cabrils, Barcelona - Spain
Contact:

Re: Redundancy with 2 bridge and LACP switchs

Thu Mar 06, 2008 6:49 pm

Hi changeip,


thanks for reply.

I think I can't make bonding with nothing, see my link in the attached JPG.


thanks,
Martín.
You do not have the required permissions to view the files attached to this post.
 
User avatar
Ibersystems
Forum Guru
Forum Guru
Topic Author
Posts: 1686
Joined: Wed Apr 12, 2006 12:29 am
Location: Cabrils, Barcelona - Spain
Contact:

Re: Redundancy with 2 bridge and LACP switchs

Thu Mar 06, 2008 6:51 pm

One more thing:

LINK must be transparent, nothing routed.

LAN1 is 192.168.1.x and LAN 2 is 192.168.1.x too.


Thanks,
Martín.
 
User avatar
nz_monkey
Forum Guru
Forum Guru
Posts: 2096
Joined: Mon Jan 14, 2008 1:53 pm
Location: Over the Rainbow
Contact:

Re: Redundancy with 2 bridge and LACP switchs

Fri Mar 07, 2008 5:39 am

Hrmmm,

Why dont you just use a single RB333 at each end with two cards installed, then run these in turbo mode with bonding ?
 
changeip
Forum Guru
Forum Guru
Posts: 3830
Joined: Fri May 28, 2004 5:22 pm

Re: Redundancy with 2 bridge and LACP switchs

Fri Mar 07, 2008 8:41 am

im guessing for bandwidth reasons ...

so the main problem is that LACP packets are not passing between the switches thru the routeros boxes? i read below (above) that its the linux bridging protocol thats stopping this ? Is this a known limitation or a bug ? I was thinking down the road of a setup like this as well and would like to know if it can't be done ahead of time so I can investigate alternatives.
 
User avatar
Ibersystems
Forum Guru
Forum Guru
Topic Author
Posts: 1686
Joined: Wed Apr 12, 2006 12:29 am
Location: Cabrils, Barcelona - Spain
Contact:

Re: Redundancy with 2 bridge and LACP switchs

Fri Mar 07, 2008 2:27 pm

Nz_monkey,

If I use only one RB333 on each side and this RB333 fails, my link fails too. I need to use 2 RB333 on each side for redundancy, also for better bandwidth.

I think that my schema/diagram isn't a strange redundancy solution. I need to have all electronics duplicated.


changeip I can confirm this. If I put 2 ethernet wires between my 2 switches configured with LACP, ports are syncronized. If I make this simulated ethernet wire with WDS link or with Nstreme link or AP-station or with EoIP tunnels ports aren't syncronized.

I think that if Mikrotik makes LACP working on RouterOS we can make better solutions. We will have support for this or anything to make redundancy 100%?



Thanks,
MArtín.
 
User avatar
promind
Member Candidate
Member Candidate
Posts: 153
Joined: Thu Feb 23, 2006 11:26 am
Location: Rousse, Bulgaria
Contact:

Re: Redundancy with 2 bridge and LACP switchs

Fri Mar 07, 2008 5:35 pm

Nz_monkey,

If I use only one RB333 on each side and this RB333 fails, my link fails too. I need to use 2 RB333 on each side for redundancy, also for better bandwidth.

I think that my schema/diagram isn't a strange redundancy solution. I need to have all electronics duplicated.


changeip I can confirm this. If I put 2 ethernet wires between my 2 switches configured with LACP, ports are syncronized. If I make this simulated ethernet wire with WDS link or with Nstreme link or AP-station or with EoIP tunnels ports aren't syncronized.

I think that if Mikrotik makes LACP working on RouterOS we can make better solutions. We will have support for this or anything to make redundancy 100%?



Thanks,
MArtín.
Let me see if I got the point...
side1: RB333 -> side2:RB333
side3: RB333 -> side4:RB333
Symmetric bonding:
RB600_1+side1+side3
RB600_2+side2+side4
side1 plugged via rj45 on ether1 of rb600_1 and side3 on ether2
side2 plugged in ether1 of rb600_2 and side4 on ether2.

rb333s wds bridge all
rb600_1 add bonding interface(use 802.3af if smartswitch behind) -> bond ether1+ether2/see manual on the wiki/
rb600_2 add bonding interface -> bond ether1+ether2
create bridge and add ports ether3+bonding interface on both sides.
u r done. :)
You do not have the required permissions to view the files attached to this post.
 
User avatar
Ibersystems
Forum Guru
Forum Guru
Topic Author
Posts: 1686
Joined: Wed Apr 12, 2006 12:29 am
Location: Cabrils, Barcelona - Spain
Contact:

Re: Redundancy with 2 bridge and LACP switchs

Fri Mar 07, 2008 5:58 pm

Hi promind,

I have to avoid using more equipement, due to redundancy of all components. If you see, there is no redundancy in RB600s. What is the problem if RB600 die?.. This is solved using LACP but I can't use here.


Thanks for your reply, it is very clear.


Martín.

Edit: If my costumer thinks that this is ok we will make this solution. This is the second best solution I think, the best solution is to use LACP.
Thanks.
 
User avatar
promind
Member Candidate
Member Candidate
Posts: 153
Joined: Thu Feb 23, 2006 11:26 am
Location: Rousse, Bulgaria
Contact:

Re: Redundancy with 2 bridge and LACP switchs

Fri Mar 07, 2008 9:04 pm

Hi promind,

I have to avoid using more equipement, due to redundancy of all components. If you see, there is no redundancy in RB600s. What is the problem if RB600 die?.. This is solved using LACP but I can't use here.


Thanks for your reply, it is very clear.


Martín.

Edit: If my costumer thinks that this is ok we will make this solution. This is the second best solution I think, the best solution is to use LACP.
Thanks.
Hi Martin,
There should be any solution...probably if using lacp over vlan...but right now I don't have the equipment to confirm if it's possible.
I've found this:
http://www.juniper.net/techpubs/softwar ... fig19.html
 
tprice42
just joined
Posts: 11
Joined: Thu Nov 22, 2007 4:25 am

Re: Redundancy with 2 bridge and LACP switchs

Wed Mar 19, 2008 10:20 pm

We recently had the same problem while trying to acheive the same result. Complete failover, higher capacity, efficient frequency useage. We were implementing this using SMC8024L2 switches, 2 x x86 dc powered rackmount MT units on each end using Ubiquiti XR5 cards into Andrew dual polarisation dishes with nstream over 50-100km distances.

What we found with this was that the link failed over perfectly when the wireless dropped. The problem was that the switches didn't register when the link came back up again and a packet storm ensued. The only way to make the switch re-negotiate the LACP was to change the port status on the routerboard via a script.

For instance setup a script on a schedule to check the registration table and if the contents changes then change the mode of ether1 from autonegotiate to 10Mbit and back again. You cannot simply disable and re-enable the the ether1 port as it doesn't turn off the port only blocks the traffic so the switch doesn't detect the status change.

The other way that a LACP type implementation can be acheived is to use Cisco switches at each end and use Cisco's 'etherchannel' technology. Because this is proprietry Cisco technology the etherchannel negotiation packets pass through the MT bridge ok. This implementation is not as good as LACP because it's slower (up to 30s to reconfigure) but at least works.

This information may be a little out of date as the last time i tried this was when V3 was still in beta, i haven't tried this with V3 stable or using vlans, MPLS etc...
 
User avatar
Ibersystems
Forum Guru
Forum Guru
Topic Author
Posts: 1686
Joined: Wed Apr 12, 2006 12:29 am
Location: Cabrils, Barcelona - Spain
Contact:

Re: Redundancy with 2 bridge and LACP switchs

Wed Mar 19, 2008 10:57 pm

Hi,


your LACP packets cross the MT link?

If LACP packets cross over the link, you don't need to make nothing else.


I don't know about your LAN scenario. : /

Can you give me more information please?


Thanks,
Martín
 
tprice42
just joined
Posts: 11
Joined: Thu Nov 22, 2007 4:25 am

Re: Redundancy with 2 bridge and LACP switchs

Thu Mar 20, 2008 1:14 am

Nothing much to say really, perfectly standard Layer 2 lan environment. The success or failure of LACP depends alot on the switch and how it is implemented. How did anyone initially establish that the LACP packets weren't transported by the Linux Bridge in MT anyway?

We initially tested this on a bench doing an uninterrupted transfer between two workstations over this link. The way that LACP, etherchannel, port trunking, etc works of course is that destination mac addresses are mapped to one port of a LACP group. So essentially there is potential for multiplying the speed of a single link within a LACP group but only if there are multiple mac addresses on either end utilizing the full potential bandwidth of the LACP group.

In our test case we had only one workstation on each end and could saturate one individual radio pair. Disabling those radios on the wireless side caused the traffic flow between workstations to immediately fail over to the remaining active radio pair. Re-enabling the disabled radio pair immediately caused a broadcast storm.

Our assumption from this testing is that the initial setup of LACP is triggered by the switch's port state change, during the LACP session the state is monitored in a way that MT doesn't interfere with and fails over properly.

I think that the assumption (probably correctly) of the switch software developers is that LACP trunks will be physically connected from switch to switch via ethernet / fibre. As soon as you add a device in the middle that masks this behavior (however transparent it might be), you yourself interfere with the expected behavior during physical hardware failure.
 
User avatar
Ibersystems
Forum Guru
Forum Guru
Topic Author
Posts: 1686
Joined: Wed Apr 12, 2006 12:29 am
Location: Cabrils, Barcelona - Spain
Contact:

Re: Redundancy with 2 bridge and LACP switchs

Wed Mar 26, 2008 8:14 pm

Hi all,

tprice42, it's possible to pass VLAN packets over the air. I done this with 2 Alvarion BreezeNET B-100 links. The 2 links are bridging traffic at the same time, not in backup mode. When one link fails all traffic pass to the other link and when the link is up again traffic goes more or less half/half.

Continuing with promind scenario, with 2 RB600 and 4 RB333, it works. Now I have rudundant link with 2 bonded links. I pass vlan tagged data with this bonding too without problems.. but.. I can't access directly to RB333s with winbox.

I think the problem is that RB600-RB600 EoIP tunnels and bonding interface makes my intermediate RB333s too transparent.. I only can access this RB333s with MAC telnet from RB600.

In my RB600s I have ether1 and bonding interface of ether2&3 in a bridge. If I put one RB600 ether2 in that bridge I can access by IP with winbox but this makes duplicated entries in IP neighbours. Entries in ether2 and same entries in bridge.

How can I solve this access?


Thanks,
Martín.
 
User avatar
Ibersystems
Forum Guru
Forum Guru
Topic Author
Posts: 1686
Joined: Wed Apr 12, 2006 12:29 am
Location: Cabrils, Barcelona - Spain
Contact:

Re: Redundancy with 2 bridge and LACP switchs

Wed Mar 26, 2008 8:19 pm

¬¬u

I disabled the duplicated entries interface in discovery interfaces.

Solved!
 
tprice42
just joined
Posts: 11
Joined: Thu Nov 22, 2007 4:25 am

Re: Redundancy with 2 bridge and LACP switchs

Wed Mar 26, 2008 11:34 pm

We ended up using two ethernet interfaces per device to solve this, one for the bridge and one for management. Ended up running 800+ metres of outdoor cat5 because of it.
 
User avatar
Ibersystems
Forum Guru
Forum Guru
Topic Author
Posts: 1686
Joined: Wed Apr 12, 2006 12:29 am
Location: Cabrils, Barcelona - Spain
Contact:

Re: Redundancy with 2 bridge and LACP switchs

Thu Mar 27, 2008 11:46 am

what?
 
tprice42
just joined
Posts: 11
Joined: Thu Nov 22, 2007 4:25 am

Re: Redundancy with 2 bridge and LACP switchs

Thu Mar 27, 2008 12:21 pm

The contract job that we did this on was one that involved 3 links all with fully redundant hardware. Most of the sites we could use indoor equipment and relatively short coax runs to the dishes but one site here: Image the radios were ourdoor mounted 60 meters up this tower which was 40 meters from the comms building. 4x radios with 2 ethernet feeds each = 800 metres of outdoor cat5.
 
User avatar
Ibersystems
Forum Guru
Forum Guru
Topic Author
Posts: 1686
Joined: Wed Apr 12, 2006 12:29 am
Location: Cabrils, Barcelona - Spain
Contact:

Re: Redundancy with 2 bridge and LACP switchs

Thu Mar 27, 2008 12:45 pm

LOL.

This is a solution but as you see, RB600 has 3 ether.
We use ether1 for administration and ether2 & 3 for the interface bonding.

All hardware in interface bonding (between RB600s) is not accessible if I don't put one RB600 ethernet in the bridge.


Martín
 
tprice42
just joined
Posts: 11
Joined: Thu Nov 22, 2007 4:25 am

Re: Redundancy with 2 bridge and LACP switchs

Thu Mar 27, 2008 12:56 pm

I was talking more along the lines of using additional ethernet interfaces on your RB333's so that you can see them, otherwise yes they should be invisible within the bonding of the RB600 interfaces. The problem that we found with this was that arp discovery didn't work properly because an arp request packet would only go down one side of the channel bonding. If it found the radio in that side it would respond but there was never any guarantees of being able to see any of the radios because of this. Better to assume that they will all be invisible and communicate with them through the back door.

Anyway i think this is getting a little off topic. Anybody got any further info on LACP protocol packets being dropped within the MT linux bridge?
 
kujtimhajredini
newbie
Posts: 30
Joined: Fri May 28, 2004 4:56 pm
Location: Prishtina
Contact:

Re: Redundancy with 2 bridge and LACP switchs

Thu Mar 27, 2008 7:52 pm

I think that the issue of failover can be done with RSTP. Prior to some version of 3.1 (the one rc8 if i am not wrong) had a bug that did not pass the RSTP packets from the switches on both sides eaven that on MT bridge stp protocol-mode was set to none. I have test this with Cisco catalyst switches and the failover works fine.
To load balance the links in equal way is to hard to do. You can balance different VLANs and play with priority on trunk interface, but load balancing for example per packet was hard to achieve.
Maybe there is a chance with Etherchannel to do it but have not tried.
 
User avatar
Ibersystems
Forum Guru
Forum Guru
Topic Author
Posts: 1686
Joined: Wed Apr 12, 2006 12:29 am
Location: Cabrils, Barcelona - Spain
Contact:

Re: Redundancy with 2 bridge and LACP switchs

Thu Mar 27, 2008 8:48 pm

I configured this load balancing with redundancy. Tomorrow it will be working at my costumer. I tell you if all works fine. I don't know if VoIP will work fine with this. I hope all works fine : D!
 
User avatar
taglio
Frequent Visitor
Frequent Visitor
Posts: 99
Joined: Wed Nov 12, 2008 4:55 pm
Location: Redama, Barcelona, Spain
Contact:

Re: Redundancy with 2 bridge and LACP switchs

Thu Oct 22, 2009 4:50 pm

Sorry for reopen an old POST, but....

It's sure that it's not possible that want to do Ibersystems in this POST???

(I'm trying it with two Netgear GS108T with last firmware and 4 RB433AH with 4.1 firmware).

Thank you.
 
User avatar
ne0031
Member Candidate
Member Candidate
Posts: 100
Joined: Tue Apr 21, 2009 10:29 pm

Re: Redundancy with 2 bridge and LACP switchs

Thu Oct 22, 2009 6:25 pm

Use a switch that also supports lldp and you won't have any issues with the port state.
 
User avatar
taglio
Frequent Visitor
Frequent Visitor
Posts: 99
Joined: Wed Nov 12, 2008 4:55 pm
Location: Redama, Barcelona, Spain
Contact:

Re: Redundancy with 2 bridge and LACP switchs

Fri Oct 23, 2009 3:20 pm

Ok i've done it and it works without more hardware! Only the one in the original thread of Ibersystem:

Image


Now i want to share some information with you, mikrotik users.

1) I can see loadbalancing working with LACP only when i test with more than one host in LAN1.
2) Loadbalancing is not 50%/50%, it's more a 80%/20%. Do you have some best result in your experience?
3) I have configured LACP in the two switches and LLDP [the two in active mode, is this correct??]
4) When i use discovery with winbox i can see only two of the four RB that form the wifi-bridge.
5) I've build the wifi-bridge with EOIP and not WDS. Does someone have experience of WDS working in this scenario?
6) I've to disable in ALL the switches and bridges protocol RSTP. With RSTP enable only a channel work.
7) When i start the entire system i've got to wait like 3 o4 minutes until all go right.

Best Regards,

RG.
 
User avatar
taglio
Frequent Visitor
Frequent Visitor
Posts: 99
Joined: Wed Nov 12, 2008 4:55 pm
Location: Redama, Barcelona, Spain
Contact:

Re: Redundancy with 2 bridge and LACP switchs

Mon Oct 26, 2009 11:14 am

Nobody have to share something about this topic?
 
User avatar
Ibersystems
Forum Guru
Forum Guru
Topic Author
Posts: 1686
Joined: Wed Apr 12, 2006 12:29 am
Location: Cabrils, Barcelona - Spain
Contact:

Re: Redundancy with 2 bridge and LACP switchs

Mon Oct 26, 2009 11:54 am

Hi,

I think 4 minutes with 2-3 PC in the network is too much time.


Load balancing 80-20% could be because different type connections.
 
icecybe
just joined
Posts: 23
Joined: Thu Nov 26, 2009 1:13 am

Re: Redundancy with 2 bridge and LACP switchs

Thu Nov 26, 2009 1:48 am

hy pple.
Please help me with a problem. I have 2 different links, one on mikrotik 411 AH, and other on rockets. The problem is latency on rocket. Mikrotik don't handle all traffic. I have also a 2 routerboard 750. Can i redirect low ping aplications thru mikrotik link and other to rocket link with that routerboards. And how?
THanks
 
dstncl
just joined
Posts: 12
Joined: Wed Dec 11, 2013 6:41 pm

Re: Redundancy with 2 bridge and LACP switchs

Thu Dec 12, 2013 1:55 pm

Dear Ibersystems,
Dear Taglio,

i'm trying to do what i think you have done, but each time i put ethernets port in a TRUNK LACP i loose the connection with the Bridge..

Can you help me, please?

I also try to follow your suggestion... but at the moment the situation is not very sunny...

thanks for the support,

nicola
 
User avatar
Ibersystems
Forum Guru
Forum Guru
Topic Author
Posts: 1686
Joined: Wed Apr 12, 2006 12:29 am
Location: Cabrils, Barcelona - Spain
Contact:

Re: Redundancy with 2 bridge and LACP switchs

Fri Dec 13, 2013 10:22 am

Who is online

Users browsing this forum: Bing [Bot], sybadi, vkp and 26 guests