Community discussions

MikroTik App
 
OriiOn
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 88
Joined: Thu Feb 25, 2010 11:54 am

Mikrotik CRS326-24S+2Q+RM

Thu Feb 16, 2023 1:55 pm

I am planing to buy this switch, to use it to connect 3x ESX hosts and a NetApp Filer via 10Gbit. Is this switch potent enough to operate all of its 24x 10Gbit ports "non-blocking" at full speed? I won't need all if its ports, but just to make sure.
Also I plan to use RouterOS instead of SwitchOS - will there be a performance hit by doing so?

The actual setup is, I want to use two switches - for redundancy purposes. They are both connected to each other with 1x 10Gbit. Each ESX host will use 4x 10Gbit "lanes/cables/ports" - two will go to the first switch, and two to the second switch. The ESX host itself has two different network cards with 2x 10Gbit ports each. So, should any piece of hardware fail (the network card in the ESX host, or a switch), network connectivity should remain.

Is this setup feasible?
 
User avatar
pcunite
Forum Guru
Forum Guru
Posts: 1345
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Mikrotik CRS326-24S+2Q+RM

Thu Feb 16, 2023 8:35 pm

It tells you in the brochure. What else do you need to know? RouterOS is fine as long as you don't try to use the CRS as a router. It is setup for switching. To understand your needs, you need to note the ports in use, direction of traffic, and the speed.

Two 40 Gbps QSFP+ ports and twenty four 10 Gbps SFP+ ports, which provides total non-blocking throughput of 320 Gbps and switching capacity of 640 Gbps with forwarding rate of 252 Mpps with most common packets
 
un9edsda
Frequent Visitor
Frequent Visitor
Posts: 76
Joined: Sun Mar 15, 2020 11:11 pm

Re: Mikrotik CRS326-24S+2Q+RM

Fri Feb 17, 2023 3:46 pm

The actual setup is, I want to use two switches - for redundancy purposes. They are both connected to each other with 1x 10Gbit. Each ESX host will use 4x 10Gbit "lanes/cables/ports" - two will go to the first switch, and two to the second switch. The ESX host itself has two different network cards with 2x 10Gbit ports each. So, should any piece of hardware fail (the network card in the ESX host, or a switch), network connectivity should remain.

Is this setup feasible?
When planning your network please keep in mind that according to the current documentation Multi-chassis Link Aggregation Group (MLAG) "is not compatible with L3 hardware offloading. When using MLAG, the L3 hardware offloading must be disabled."
 
OriiOn
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 88
Joined: Thu Feb 25, 2010 11:54 am

Re: Mikrotik CRS326-24S+2Q+RM

Wed Feb 22, 2023 3:28 pm

Thank you for pointing out the MLAG situation! Two questions regarding MLAG:

1) What exactly does using MLAG mean, respectively what's the downside of not using MLAG, and treating the 2 switches as independent individuals? If I understand it correctly, MLAG means that a connected client sees the switches as one physical unit. What's the advantage of that? In case one switch fails there is a seamless continuation of network connectivity without any downtime/delay what so ever? Ease of management, since only "one" switch needs to be configured/monitored/maintained? Is there any performance hit when NOT using MLAG?

2) Although we intend to run RouterOS on it, we do not intend to use any routing functionality on these two switches, but use it as pure-hardware switches only. Using RouterOS is only out of convenience, since we are familiar with RouterOS and know how to maintain/monitor it remotely. AFAIK there is no downside in using RouterOS, right? With that in mind, I assume it is fine to use the two switches in our setup with RouterOS running in MLAG mode?
 
User avatar
sirbryan
Member
Member
Posts: 304
Joined: Fri May 29, 2020 6:40 pm
Location: Utah
Contact:

Re: Mikrotik CRS326-24S+2Q+RM

Wed Feb 22, 2023 6:08 pm

Yes, MLAG means that LACP connections from one ESXi host split across switches will appear (to ESXi) to be going back to a single switch and will load-balance across both switches.

There is no performance decrease using Router OS if you're just switching. My CRS300 (305,309,310,317,326,328) switches all hum along on RouterOS 7 with hardly any CPU load on them.
 
roadracer96
Forum Veteran
Forum Veteran
Posts: 730
Joined: Tue Aug 25, 2009 12:01 am

Re: Mikrotik CRS326-24S+2Q+RM

Sat Feb 25, 2023 5:13 am

Dont buy it if you intend to use MLAG. MLAG is severely broken in Mikrotik and they have demonstrated zero interest in fixing it.. Lots of talk.. no action.. Ive been sitting on a pair of CRS317-1G-16S+ for over a year now and MLAG is still as broken (more so) than it has ever been.
 
User avatar
mkx
Forum Guru
Forum Guru
Posts: 11452
Joined: Thu Mar 03, 2016 10:23 pm

Re: Mikrotik CRS326-24S+2Q+RM

Sat Feb 25, 2023 12:17 pm

Ease of management, since only "one" switch needs to be configured/monitored/maintained? Is there any performance hit when NOT using MLAG?

Nope, MLAG has nothing to do with switch management. Corporate type of switches use some sort of stacking to create a larger "virtual" switch out of smaller units (stacking includes a "back plane" connection which allows high-speed inter-switch communication and a proprietary protocol for central nanagement of the whole stack). Mikrotik doesn't have stacking functionality, the closest to it comes port extender. Sadly it's mostly about centralized managenent at cost of performance.
 
OriiOn
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 88
Joined: Thu Feb 25, 2010 11:54 am

Re: Mikrotik CRS326-24S+2Q+RM

Tue Feb 28, 2023 12:21 pm

Thanks @roadracer96
Is it true that MLAG is unusable in Mikrotik Switches in it's current state? Can other users confirm that?
So are you running the switches without MLAG instead?
I wonder if @sirbryan is using MLAG, because I have the impression (from his post) that he is happy with the switches?
What's other users experience with MLAG on CRS switches?

We only intend to use these 2 switches to hook up 3x ESXi's with a NetApp filer. What is important to us is reliable connectivity (especially in case one device should fail), full speed of the 10Gbit ports. If on top of that we take advantage of LACP with "increased liability" and load balancing, that's a plus of course.

Easy of management (stacking, "manage as one device") is NOT important to us, since there's only these 2 switches to manage.
 
roadracer96
Forum Veteran
Forum Veteran
Posts: 730
Joined: Tue Aug 25, 2009 12:01 am

Re: Mikrotik CRS326-24S+2Q+RM

Tue Feb 28, 2023 9:52 pm

Mlag doesn't fail over transparently. If the primary device restarts or fails. The lacp system id and STP bridge ID of the secondary reverts causing complete outage for several seconds.

It works. Unless you want it to work for faiover. Not even close. It's half assed.

Spend the money and get a good switch If you need this functionality.
 
mfedotov
just joined
Posts: 18
Joined: Mon Oct 25, 2021 3:32 am

Re: Mikrotik CRS326-24S+2Q+RM

Wed Mar 01, 2023 6:36 pm

Hi @OriiOn

While MLAG/VCP is obviously better, people have used the redundant switches without MLAG support for long time.
If you use iSCSI for connectivity to NetApp, there should be no issue at all, you create multi-path configuration on two different subnets. For NFS and VM traffic you would use port groups with multiple uplinks connected to the different switches. NetApp can do similar configuration using the broadcast domains/failover groups.
 
roadracer96
Forum Veteran
Forum Veteran
Posts: 730
Joined: Tue Aug 25, 2009 12:01 am

Re: Mikrotik CRS326-24S+2Q+RM

Sat Mar 04, 2023 3:28 am

Hi @OriiOn

While MLAG/VCP is obviously better, people have used the redundant switches without MLAG support for long time.
If you use iSCSI for connectivity to NetApp, there should be no issue at all, you create multi-path configuration on two different subnets. For NFS and VM traffic you would use port groups with multiple uplinks connected to the different switches. NetApp can do similar configuration using the broadcast domains/failover groups.
Keep in mind. Using port groups in vmware breaks some networking. It's the worst idea anyone ever came up with and for some reason people still cling to using it. 2 words. Multicast and promiscuous mode. Mlag/lacp fixes this.
 
OriiOn
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 88
Joined: Thu Feb 25, 2010 11:54 am

Re: Mikrotik CRS326-24S+2Q+RM

Mon Mar 06, 2023 4:03 pm

Thanks for your replies!

@roadracer96: so which MLAG/LACP compliant switch would you recommend? The ones I am seeing from Cisco or HP cost €5000 and up - per piece. Compared to what we spent on the 3 ESXi's that seems unproportional. Ok, the NetApp filer did cost more (€20.000). But it seems unreasonable to spend €10K for two switches just to connect three machines? Or am I unrealistic? In other words, are there MLAG/LACP switches at around €2K per piece that suits our needs?

And I agree, I would like to avoid having to do complicated setups on the ESXi/VM's and/or NetApp filer, if there IS a technology (MLAG/LACP) that takes care of reliability and load-balancing nicely.

Who is online

Users browsing this forum: akira463 and 35 guests