Any hardware recommendations for proxmox cluster with 20GBit/s inter-VLAN ip filtering?

RouterOS Beginner Basics
1

Hi all

First of all: I’m new to MikroTik and never used one before :wink:

My goal is to set up the following network:


                            MikroTik switch/router
                                     |
                  2x10Gbit/s (LACP)  |
   |--- proxmox1 =================== |
 Ceph-- proxmox2 =================== |
   |--- proxmox3 =================== |
                                     | ---- ISP uplink (10 or 25Gbit/s)
                                     | ---- LAN (Access Points, etc.)
                                     |

Further informations:

  • I want to use a virtual OPNsense on the proxmox cluster as firewall/router/DNS server/DHCP server/etc.
  • All network segregation will be done with VLANs
  • If possible I want to use some inter-VLAN ip filtering/routing on the switch to not route all traffic over OPNsense

My questions are:

  1. Is this network design useful? Any improvements/comments?
  2. Which hardware do you prefer for such a use case?
  • Fasttrack L3HW for inter-VLAN ip filtering available?
  • Sufficient performance to inter-Proxmox/VLAN transfer 20GBit/s?
  • Examples:
  • CCR2004-1G-12S+2XS
    • CRS317-1G-16S+RM
    • CRS326-4C+20G+2Q+RM
    • Other?
  1. Which OS? RouterOS or SwOS?

Thanks in advance for all comments!

2

Old thread, but I use the CRS317 with 10Gb SFP+ modules in LACP 802.3ad bonded to the servers which run Intel X520-DA2 NIC’s with 2 ports as trunk ports to handle the tagging. I use both XCP-ng and Proxmox clusters for the hypervisor functions. However I use an rb5009 for all routing, trunked to the CRS317 - so my setup is a bit different in that I don’t use pfsense. I run RouterOS on both devices, version 7.16, all runs fine for my needs. I host about a dozen cloud workloads over it, and I need the 10G links for the shared storage VLAN mostly. I run the corosync network over a standard 1GB uplink.
Best of luck on your setup.

3

Dear Team,
I am facing some issue with CCR-1072
loading kernel… OK
setting up elf image… OK
jumping to kernel code
(0,0) hv_panic: msh3: detected uncorrectable memory error on rank 0 (H)
System halted.
(6,1) hv_panic: got read error response on RDN interrupt: PC 0xffff_fffc_0003_8e88, ICS/PL 0x3

4

Looks like it is ready for the trash-bin then.
The CCR1072 has 16GB embedded (1-bit error correcting memory) ECC memory, so if the kernel sees an “uncorrectable error” I think it really busted
As far as I know you cannot just replace memory-modules.