Community discussions

MikroTik App
 
User avatar
metalcated
just joined
Topic Author
Posts: 17
Joined: Fri Apr 19, 2013 3:07 pm
Contact:

How to add second CRS326-24G-2S+RM with complex VLAN setup

Wed Mar 29, 2023 3:58 am

Hey all.

So, buddy of mine is a network guru and setup my gear at home for me and I understand it for the most part. However, I would consider some of it to be complex for someone like me who is more of a systems admin vs. network admin. He went poof about a year ago and I have no idea what happened to him, that is a different topic of discussion.

Now to the topic at hand:

Current setup
1x RB4011iGS+ (Router)
1x CRS326-24G-2S+RM (Switch 24 port)
1x HP ProCurve 24 Port switch
1x Netgear ProSafe 24 Port switch
1x cAP AP and 3 Ubiquiti AP's
3x ESXi hosts (1x Dell R910 / 2x R620)
2x SAN's (1x Dell R330 and 1x Supermicro home build)
1x 1U APC Battery Backup Unit

I have multiple VLAN's
10 - Management
11 - Client (DHCP)
12 - Guest (DHCP)
13 - Eoip over VPN (not setup by me so I don't have a clue how to manage it)
15 - Storage (ESXi hosts -> SANs for iSCSI)
16 - Family (Kids network using different DNS hosts etc. and is more restricted)
17 - IoT devices
20 - Voice for PBX phones

I could keep going on the entire setup but I think that gives enough background to begin this conversation.

I am adding a second CRS326-24G-2S+RM and removing the HP ProCurve. I am taking the Netgear switch and putting it where the ProCurve is (second floor of the house).
I installed the second switch and plugged in a 10G DAC cable, seems to be functioning correctly as I am seeing movement on the interface on Switch #1. I tried duplicating the setup from Switch #1 to Switch #2 but lost connectivity after setting the "bridge1" bridge the same as Switch #1. I wasn't sure what would happen if I did that but lost complete and total connectivity resulting in a reset.

Bridge1 setting that resulted in total connectivity loss:
Greenshot 2023-03-28 20.48.57.png
Confirmation of traffic and bridge1 interface snippet:
Greenshot 2023-03-28 20.55.50.png
I have read that you can basically add the second switch and leverage the existing setup on the first switch but I don't see any information on how that is done. I imagine an export of my config is required so decided to post here but wanted to see if anyone has any links or information they can share to help me out and before I post the config (sanitized).
You do not have the required permissions to view the files attached to this post.
 
User avatar
mkx
Forum Guru
Forum Guru
Posts: 11381
Joined: Thu Mar 03, 2016 10:23 pm

Re: How to add second CRS326-24G-2S+RM with complex VLAN setup

Wed Mar 29, 2023 8:22 am

Before you enable VLAN filtering on bridge, you have to set up the rest of VLAN settings:
  • VLANs on trunk port (connecting towards core of network)
  • vlan interface for management
  • set IP address on vlan interface for management
And only then enable VLAN filtering.

Could be that MAC connectivity remains working after you (prematurely) enable VLAN filtering, try to connect to switch using winbox - if winbox shows switch, then click its MAC address.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: How to add second CRS326-24G-2S+RM with complex VLAN setup

Wed Mar 29, 2023 2:42 pm

Step 1 - take a port of the bridge to use for configuration purposes give it an IP address liek 192.168.55.1/24 and add it to a management interface list, give your laptop an iPV4 address such as 192.168.55.5 and you are in. That way smooth sailing during bridge and vlan changes!!
viewtopic.php?t=181718

One bridge ( and no dhcp or anything else for bridge )
VLANs Use separate management vlan to tie all smart devices downstream with an IP address on this VLAN
(neighbours discovery etc. )

Trunk port to first switch carrying all vlans
 
User avatar
metalcated
just joined
Topic Author
Posts: 17
Joined: Fri Apr 19, 2013 3:07 pm
Contact:

Re: How to add second CRS326-24G-2S+RM with complex VLAN setup

Wed Mar 29, 2023 7:34 pm

Before you enable VLAN filtering on bridge, you have to set up the rest of VLAN settings:
  • VLANs on trunk port (connecting towards core of network)
  • vlan interface for management
  • set IP address on vlan interface for management
And only then enable VLAN filtering.

Could be that MAC connectivity remains working after you (prematurely) enable VLAN filtering, try to connect to switch using winbox - if winbox shows switch, then click its MAC address.
Yeah I did all of that before enabling filtering and I also was connected to the MAC instead of the IP. Even then I was still unable to connect after enabling filtering.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: How to add second CRS326-24G-2S+RM with complex VLAN setup

Wed Mar 29, 2023 8:06 pm

and the reason not to provide the config/evidence on RB4011 is?????
 
User avatar
metalcated
just joined
Topic Author
Posts: 17
Joined: Fri Apr 19, 2013 3:07 pm
Contact:

Re: How to add second CRS326-24G-2S+RM with complex VLAN setup

Wed Mar 29, 2023 9:01 pm

and the reason not to provide the config/evidence on RB4011 is?????
Work is busy? I am going to upload as soon as I can.
 
User avatar
metalcated
just joined
Topic Author
Posts: 17
Joined: Fri Apr 19, 2013 3:07 pm
Contact:

Re: How to add second CRS326-24G-2S+RM with complex VLAN setup

Wed Mar 29, 2023 10:41 pm

Heavily stripped down version of the
/export
config. I removed the firewall rules, let me know if you need to see those as well.
rtr01] > /export
# mar/29/2023 14:39:24 by RouterOS 6.49.5
# software id = GBDC-2Q6I
#
# model = RB4011iGS+
/interface bridge
add name=bridge1
/interface ethernet
set [ find default-name=ether4 ] auto-negotiation=no comment="To sw01 interface 2"
set [ find default-name=ether5 ] auto-negotiation=no comment="To sw01 interface 3"
set [ find default-name=ether6 ] auto-negotiation=no comment="To sw01 interface 4"
set [ find default-name=ether7 ] auto-negotiation=no comment="To sw01 interface 5"
set [ find default-name=ether8 ] name=ether8
set [ find default-name=ether9 ] name=ether9
set [ find default-name=ether1 ] name=wan1
/interface bonding
add comment=SW01 link-monitoring=none mode=802.3ad name=LAG1 slaves=ether4,ether5,ether6,ether7
/interface vlan
add interface=LAG1 name=vlan10-management vlan-id=10
add interface=LAG1 name=vlan11-clients vlan-id=11
add interface=LAG1 name=vlan12-guest vlan-id=12
add interface=LAG1 name=vlan13-eoip-s2s-cadc vlan-id=13
add interface=LAG1 name=vlan15-storage vlan-id=15
add interface=LAG1 name=vlan16-family vlan-id=16
add interface=LAG1 name=vlan17-IoT vlan-id=17
add interface=LAG1 name=vlan20-voice vlan-id=20
/interface ethernet switch port
set 0 default-vlan-id=0
set 1 default-vlan-id=0
set 2 default-vlan-id=0
set 3 default-vlan-id=0
set 4 default-vlan-id=0
set 5 default-vlan-id=0
set 6 default-vlan-id=0
set 7 default-vlan-id=0
set 8 default-vlan-id=0
set 9 default-vlan-id=0
set 10 default-vlan-id=0
set 11 default-vlan-id=0
/interface list
add name=MGMT
/ip pool
add name=voice-dhcp-pool ranges=1.2.3.163-1.2.3.174
add name=client-dhcp-pool ranges=1.2.3.66-1.2.3.125
add name=guest-dhcp-pool ranges=1.2.3.130-1.2.3.158
add name=family-dhcp-pool ranges=1.2.3.196-1.2.3.222
add name=IoT-dhcp-pool ranges=1.2.3.226-1.2.3.254
add name=storage-dhcp-pool ranges=1.2.3.181
/ip dhcp-server
add address-pool=voice-dhcp-pool authoritative=after-2sec-delay conflict-detection=no disabled=no interface=vlan20-voice lease-time=1d name=voice-dhcp-server
add address-pool=client-dhcp-pool authoritative=after-2sec-delay conflict-detection=no disabled=no interface=vlan11-clients lease-time=1d name=clients-dhcp-server
add address-pool=guest-dhcp-pool authoritative=after-2sec-delay interface=vlan12-guest lease-time=1d name=guests-dhcp-server
add address-pool=family-dhcp-pool conflict-detection=no disabled=no interface=vlan16-family lease-time=1d name=family-dhcp-server
add address-pool=IoT-dhcp-pool disabled=no interface=vlan17-IoT lease-time=1d name=iot-dhcp-server
add address-pool=storage-dhcp-pool disabled=no interface=vlan15-storage lease-time=1d name=storage-dhcp-server
/interface bridge port
add bridge=bridge-eoip-s2s-cadc interface=vlan13-eoip-s2s-cadc
add bridge=bridge-eoip-s2s-cadc interface=eoip-s2s-cadc
/ip neighbor discovery-settings
set discover-interface-list=MGMT
/interface bridge vlan
add bridge=bridge1 comment="VLAN 10 - MGMT" tagged=bridge1 vlan-ids=10
add bridge=bridge1 comment="VLAN 11 - CLIENTS" tagged=bridge1 vlan-ids=11
add bridge=bridge1 comment="VLAN 17 - IOT" tagged=bridge1 vlan-ids=17
add bridge=bridge1 comment="VLAN 20 - VOICE" tagged=bridge1 vlan-ids=20
add bridge=bridge1 comment="VLAN 16 - FAMILY" vlan-ids=16
add bridge=bridge1 comment="VLAN 15 - STORAGE" vlan-ids=15
/interface detect-internet
set detect-interface-list=all
/interface list member
add interface=vlan10-management list=MGMT
add interface=vlan11-clients list=MGMT
/ip address
add address=1.2.3.129/27 comment="Guest (vLAN12)" disabled=yes interface=vlan12-guest network=1.2.3.128
add address=1.2.3.193/27 comment="Family (vLAN16)" interface=vlan16-family network=1.2.3.192
add address=1.2.3.225/27 comment="IoT (vLAN17)" interface=vlan17-IoT network=1.2.3.224
add address=1.2.3.33/27 comment="Management (vLAN10)" interface=vlan10-management network=1.2.3.32
add address=1.2.3.65/26 comment="Clients (vLAN11)" interface=vlan11-clients network=1.2.3.64
add address=1.2.3.177/29 comment="Storage (vLAN15)" interface=vlan15-storage network=1.2.3.176
add address=1.2.3.161/28 comment="Voice (vLAN20)" interface=vlan20-voice network=1.2.3.160
/ip cloud
set ddns-enabled=yes update-time=no
/ip dhcp-client
add disabled=no interface=wan1 use-peer-dns=no
/ip dhcp-server network
add address=1.2.3.64/26 boot-file-name=pxelinux.0 comment="client = vlan11" dns-server=1.2.3.45,4.5.6.45,1.2.3.46 domain=domain.com gateway=1.2.3.65 netmask=26 next-server=1.2.3.50 ntp-server=\
    1.2.3.45,4.5.6.45
add address=1.2.3.128/27 comment="guest = vlan12" dns-server=1.2.3.59,4.5.6.59 domain=guest.domain.com gateway=1.2.3.129 netmask=28 ntp-server=1.2.3.129
add address=1.2.3.160/28 comment="voice = vlan20" dhcp-option-set=VOICE dns-server=1.2.3.45,4.5.6.45,1.2.3.46 domain=domain.com gateway=1.2.3.161 netmask=28 ntp-server=1.2.3.1
add address=1.2.3.176/29 comment="storage = vlan15" dns-server=1.2.3.45,1.2.3.46 domain=storage-vlan15.domain.com gateway=1.2.3.177 netmask=29 ntp-server=1.2.3.177
add address=1.2.3.192/27 comment="family = vlan16" dns-server=1.2.3.59,4.5.6.59 domain=family.domain.com gateway=1.2.3.193 netmask=27 ntp-server=1.2.3.193
add address=1.2.3.224/27 comment="IoT = vlan17" dns-server=8.8.8.8,8.8.4.4,1.1.1.1,1.0.0.1 domain=iot.domain.com gateway=1.2.3.225 netmask=27 ntp-server=1.2.3.225
Do you also need an
/export
of sw01?
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: How to add second CRS326-24G-2S+RM with complex VLAN setup

Thu Mar 30, 2023 12:49 am

Why the LAG construct??? I dont know what is the correct way to approach bridge and vlans.???
Why not just push the vlans on a single trunk port to a switch.........

In any case these /interface bridge vlan entries dont do anything.....
add bridge=bridge1 comment="VLAN 16 - FAMILY" vlan-ids=16
add bridge=bridge1 comment="VLAN 15 - STORAGE" vlan-ids=15
 
User avatar
metalcated
just joined
Topic Author
Posts: 17
Joined: Fri Apr 19, 2013 3:07 pm
Contact:

Re: How to add second CRS326-24G-2S+RM with complex VLAN setup

Thu Mar 30, 2023 1:31 am

Why the LAG construct??? I dont know what is the correct way to approach bridge and vlans.???
Why not just push the vlans on a single trunk port to a switch.........

In any case these /interface bridge vlan entries dont do anything.....
add bridge=bridge1 comment="VLAN 16 - FAMILY" vlan-ids=16
add bridge=bridge1 comment="VLAN 15 - STORAGE" vlan-ids=15
If you recall in the OP, I did not set this up. You seem very aggressive in your comments as well. I would appreciate some direction, more of a step by step "this is how you accomplish what you are looking to do" approach rather than to take stabs at a config I did not setup.

I need working VLAN's and the second switch setup to work like the first. If making changes to get that done is what I need to do, then do be it but like I said, I need step by step. I am no network guru, I am here for help. Thank you.
 
User avatar
metalcated
just joined
Topic Author
Posts: 17
Joined: Fri Apr 19, 2013 3:07 pm
Contact:

Re: How to add second CRS326-24G-2S+RM with complex VLAN setup

Thu Mar 30, 2023 4:07 am

Why the LAG construct??? I dont know what is the correct way to approach bridge and vlans.???
Why not just push the vlans on a single trunk port to a switch.........

In any case these /interface bridge vlan entries dont do anything.....
add bridge=bridge1 comment="VLAN 16 - FAMILY" vlan-ids=16
add bridge=bridge1 comment="VLAN 15 - STORAGE" vlan-ids=15
Also, to comment on those not doing anything. Yes I added those a couple of days ago trying to get a Ubiquiti switch connected which I ended up sending back and buying this second Mikrotik switch instead. I WAS going to replace all of my Mikrotik switches with Ubiquiti switches but decided against it due to the lack of features on Ubiquiti. Anyways, totally different topic. All I want to do is get this all working with my router and 2 switches where it makes the most sense configuration wise. Need my VLAN's working and wanted to add LAG's for my 3 vCenter hosts and 2 SAN's.

Thanks
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: How to add second CRS326-24G-2S+RM with complex VLAN setup

Thu Mar 30, 2023 2:00 pm

viewtopic.php?t=143620
More like frustrated that you never bothered to learn about the system in your own home for someone that is trained.

As stated I would modify the config to a more standard vlan setup until I was comfortable how LAGs and vlans worked and then that would be stage 2.
YOu have big learning curve and no one here is going to spoon feed the answers so you will have to learn.
If willing then help will be here.

The first thing you should post is a decent network diagram to provide the context!!!
Most homeowners dont have half the stuff you have, 2 port LAG though is common with NAS stuff.
Do you know what the EOIP is for ?

In Summary, adding a switch to a network you dont have a clue about is the wrong course of action.
Thats down the line for now.
 
User avatar
metalcated
just joined
Topic Author
Posts: 17
Joined: Fri Apr 19, 2013 3:07 pm
Contact:

Re: How to add second CRS326-24G-2S+RM with complex VLAN setup

Thu Mar 30, 2023 3:54 pm

viewtopic.php?t=143620
More like frustrated that you never bothered to learn about the system in your own home for someone that is trained.

As stated I would modify the config to a more standard vlan setup until I was comfortable how LAGs and vlans worked and then that would be stage 2.
YOu have big learning curve and no one here is going to spoon feed the answers so you will have to learn.
If willing then help will be here.

The first thing you should post is a decent network diagram to provide the context!!!
Most homeowners dont have half the stuff you have, 2 port LAG though is common with NAS stuff.
Do you know what the EOIP is for ?

In Summary, adding a switch to a network you dont have a clue about is the wrong course of action.
Thats down the line for now.
I 100% appreciate your stance on the overall topic. I do have a general understanding on networking and VLAN's its more of the how does RouterOS do it the right way. (I will do some reading but would like to have a dummies summary instead of an in-depth read, regardless I will look it over - thank you).
Second, I already have the sw02 in place and plugged in. I was able to get it on the network last night talking over VLAN10 (MGMT) but nothing else works at the moment. I do believe I am at the stage where I need to tick that box on the bridge to enable vlan filtering but didn't want to do it just yet in fear of loosing communication again. I will use Safe Mode but still and hesitant at the moment. I believe the EOIP was setup as a test by my buddy to see if I was able to obtain IP's from a DHCP network in a remote DC which is connected over VPN. It works but I have no use for it besides maybe as a backup/recovery method if something goes wrong with one of the VM's that I am backing up from that DC to my home lab. Could be useful in that scenario. LAG's I understand, not overly complicated to setup and not 100% necessary since 1G is 1G. I eventually will be upgrading to 10G. I think my mindset of what a LAG does and what it actually does in a vCenter/VMware environment isn't what I initially thought. As for the NAS it would have a better application over VMWare.

Thanks for the response, I will keep posting and updating as I think of more and learn more.
 
404Network
Member Candidate
Member Candidate
Posts: 285
Joined: Wed Feb 16, 2022 2:04 pm

Re: How to add second CRS326-24G-2S+RM with complex VLAN setup

Thu Mar 30, 2023 4:22 pm

Understood especially the trepidation about engaging vlan-filtering=yes.
My work around is to avoid configuring from the bridge.
Take an empty port assign it an IP address ONLY, 192.68.55.1/24 network 192.168.55.0 interface=etherX
Ensure you add ether5 to the appropriate interface list and/or rule on input chain to ensure it has "admin access"
Then easy peasy, connect laptop to etherX put in for example 192.168.55.5 as ipv4 setting on the nic card settings and you will be
able to config the router but safely off the bridge. STILL USE SAFE MODE for all changes.

(Safe mode: invoke, make changes, wait 15 seconds, if no hiccups, un-select to make changes permanent and re-select for next set of changes)
 
User avatar
metalcated
just joined
Topic Author
Posts: 17
Joined: Fri Apr 19, 2013 3:07 pm
Contact:

Re: How to add second CRS326-24G-2S+RM with complex VLAN setup

Thu Mar 30, 2023 4:44 pm

Understood especially the trepidation about engaging vlan-filtering=yes.
My work around is to avoid configuring from the bridge.
Take an empty port assign it an IP address ONLY, 192.68.55.1/24 network 192.168.55.0 interface=etherX
Ensure you add ether5 to the appropriate interface list and/or rule on input chain to ensure it has "admin access"
Then easy peasy, connect laptop to etherX put in for example 192.168.55.5 as ipv4 setting on the nic card settings and you will be
able to config the router but safely off the bridge. STILL USE SAFE MODE for all changes.

(Safe mode: invoke, make changes, wait 15 seconds, if no hiccups, un-select to make changes permanent and re-select for next set of changes)
Yeah I was reading about that from the previous posted link. Good fail safe!

Who is online

Users browsing this forum: baragoon, GoogleOther [Bot], sokalsondha and 42 guests