Community discussions

MikroTik App
 
SilverNodashi
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 77
Joined: Mon Sep 04, 2017 4:18 pm
Location: South Africa
Contact:

redundant network setup

Tue Sep 07, 2021 12:07 pm

Hi,

I have searched the forums but cannot find a good answer. I need to setup a redundant network for a linux VM cluster.
- The Linux machines have bonded NIC's with 802.3ad running between 2x MikroTik CRS326-24G-2S+. There's a fiber crossover on the two Mikrotik's SFP ports. This works fine. When either switch goes off, the network still works the VM's still operate fine. /
* Is it possible to achieve aggregated traffic between the 2 switches?

- We have 1 internet connection at the moment, connected to a RB3011. From the RB3011 I have one cable to each CRS326-24G-2S+.
* How do I achieve proper redundancy, if either switch goes offline? I have tried 802.3ad on the RB3011 but that doesn't seem to work. Right now both switches are just in a bridge, but I am not sure if this is the right way todo it.
* Later on, when we have a 2nd uplink to the internet, I would like to have redundant WAN failover.
* If I add another RB3011, what do I need to setup, to have reduntant failover to the internet (with 1 uplink to each router), if the provider setup bonding on their side?
redundant network.PNG
You do not have the required permissions to view the files attached to this post.
 
tdw
Forum Guru
Forum Guru
Posts: 1843
Joined: Sat May 05, 2018 11:55 am

Re: redundant network setup

Wed Sep 08, 2021 8:17 pm

Trying to introduce redundancy can create more points of failure, you have to consider carefully what would happen if any device or interconnection fails and what the traffic paths are likely to be.

If you want to have active-active bonding split across switches you need MLAG, this has only recently been introduced in ROSv7 for CRS3xx devices. Otherwise use active-backup bonding where only one link is operational.

Another mechanism you could consider, especially between the router and switches is spanning tree. This blocks one of the links which would otherwise form a loop, by adjusting the RSTP bridge priorities you can ensure the blocked link is positioned in the most sensible place in your topology.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 19101
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: redundant network setup

Wed Sep 08, 2021 8:56 pm

THe weak chain in the link is a single RB3011~
(assuming the two lines from the router indicate two WAN connections)
 
SilverNodashi
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 77
Joined: Mon Sep 04, 2017 4:18 pm
Location: South Africa
Contact:

Re: redundant network setup

Wed Sep 08, 2021 10:33 pm

THe weak chain in the link is a single RB3011~
(assuming the two lines from the router indicate two WAN connections)
Right now, this is what I have at my disposal. There's only 1 uplink as well, so a 2nd RB3011 wouldn't make a difference either.

The idea is to have 2 routers, each with it's own internet uplink. IF this is at all possible, I can order another RB3011 (or even 4011 if needed) and another internet uplink. So, for now, it's a POC.
 
SilverNodashi
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 77
Joined: Mon Sep 04, 2017 4:18 pm
Location: South Africa
Contact:

Re: redundant network setup

Wed Sep 08, 2021 10:43 pm


Trying to introduce redundancy can create more points of failure, you have to consider carefully what would happen if any device or interconnection fails and what the traffic paths are likely to be.
The whole idea is to have two internet paths to each server. This way, if a router / switch / uplink / network card fails, there would still be connectivity.

If you want to have active-active bonding split across switches you need MLAG, this has only recently been introduced in ROSv7 for CRS3xx devices. Otherwise use active-backup bonding where only one link is operational.
I have looked into Mlag, but ROSv7 isn't stable / mainstream yet, though it's one of the reasons why I got the CRS326-24G-2S+ switches. So I am looking for a workable solution in the meantime. Active-backup would work as well, but I cannot figure out how to get it working. Does that mean I need to use the CRS326-24G-2S+ as a router, as well as a network switch? There would be a lot of network (non-internet) traffic between the cluster nodes. which is one reason why I would prefer to have a dedicated router(s) for internet breakout.

Another mechanism you could consider, especially between the router and switches is spanning tree. This blocks one of the links which would otherwise form a loop, by adjusting the RSTP bridge priorities you can ensure the blocked link is positioned in the most sensible place in your topology.
Can you perhaps point me in the right direction, in terms of a wiki / tutorial / write-up on how to set it up?
 
tdw
Forum Guru
Forum Guru
Posts: 1843
Joined: Sat May 05, 2018 11:55 am

Re: redundant network setup

Wed Sep 08, 2021 11:25 pm

Two internet paths is tricky if you required the same inbound addresses, unless you have separate external connections and BGP there is going to be a single point of failure.

For active-backup no bonding setup at the switch end of the server connections, just on the servers themselves - search for linux bonding active-backup.The CRS3xx are still operating as switches, if the links from the servers to .6 switch are the primary/active ones, and those to .7 switch are the backup ones then all internet and inter-server traffic will normally be handled by the .6 switch. If any of the server to .6 switch connections fail then traffic to/from that server will be via the SFP link between switches and .7 switch. If the .6 switch fails all the connections will use the .7 switch.

Whilst you could use an active-backup bond on the 3011 to do the same for connections to the two switches it wouldn't provide a path between the two switches if the SFP link failed, so RSTP would be better. Set the bridge priorities so the 3011 to .7 switch path is the least favoured link so normally internet traffic will flow through the RB3011 to .6 switch, if that link or .6 switch fails internet traffic will flow through the RB3011 to .7 switch link. If the SFP link fails both switches can still communicate via the RB3011, albeit at 1G rather than 10G.

https://help.mikrotik.com/docs/display/ ... e+Protocol, stick with RSTP and ignore MSTP as it isn't necessary for your setup.

Who is online

Users browsing this forum: cdblue, intania, tesme33 and 55 guests