vLAN with Switch chips _ scenario-based solutions

paradigm · Fri Feb 28, 2014 11:26 am

vLAN.jpg

I have a RB751-2HnD that has a Atheros7240 switch chip.
As you can see in the above picture I set master port for eth3-eth5 to eth2. so I have a 5 port switch with port:
1- port1 = cpu port
2- port2 = eth2
3- port3 = eth3
4- port4 = eth4
5- port5 = eth5

The simple scenario is:

All of my pcs have a IP address in range 192.168.1.x/24 so that in the normal situation they can communicate with each other. but
I want to have 2 vLANs, vLAN(A) and vLAN(B).
pc2 and pc3 are in vLAN(A) with vLAN tag id 200 and pc4 and pc5 are in vLAN(B) with vLAN tag id 400.
With this scenario I want to reach this goal:

The pc2 and pc3 can communicate with each other and pc4 and pc5 also too. and neither of pc2 or pc3 can not communicate with pc4 or pc5 and vise avers.

dasiu · Fri Feb 28, 2014 11:45 am

What do you mean by port1 = cpu port ??
If ether2 is the master port, then the cpu port of the switch chip is ether2 of router's CPU. I think that you're a bit confused with the terminology

.
You can check my MUM presentation about the switch chip: http://mum.mikrotik.com/presentations/I ... nowski.pdf

But OK, let's assume that ether3-ether5 have "master port" set to ether2. Then:
1. In the Switch menu, in tab called "VLAN", create vlan 200 with ports ether2 and ether3, and vlan 400 with ports ether4 and ether5.
2. In the "port" tab you edit all 4 ports ether2-ether5, set VLAN Mode to "secure", VLAN Header to "always strip", and Default VLAN ID to 200 (ether2 and ether3) or 400 (ether4 and ether5).
That way pc2 and pc3 can see each other, pc4 and pc5 can see each other, but nothing more

.

If you want the RouterBoard to be the gateway for the PCs, it's a bit more complicated:
3. In the "VLAN" tab add "cpu" port to vlans 200 and 400
4. In the "port" tab edit the cpu port, set VLAN Mode to "secure" and "VLAN Header" to "leave-as-is".
5. Create /interface vlan add name=vlan200 interface=ether2 tag=200 (and 400 the same way). You do it on ether2, as it's the master port = the CPU's port connected to the switch (see my presentation).
6. Create /interface bridge add name=bridge1
7. Add /interface bridge port add bridge=bridge1 interface=vlan200 horizon=1 (and vlan400 the same way). By setting the horizon to the same value - you make the 2 vlans separated from each other, but the MikroTik will have access to both.
8. You set the IP adress (ex. 192.168.1.1/24 - if it's the gateway address - the address on your router) on the bridge1 interface.

efaden · Fri Feb 28, 2014 1:36 pm

In the example you posted you don't need to use the switch chip at all. Can you explain what your overall goal is?

-Eric

CelticComms · Fri Feb 28, 2014 2:38 pm

What isn't clear from the original post is whether the two VLANs need to communicate with the router or (say) an ISP on Ether 1 via NAT.

If they do:

Create VLAN interfaces for VLAN 200 & VLAN 400 on Ether 2
Under Switch, create VLAN 200 and add ports "switch 1 CPU", Ether 2 & Ether 3 to it
Under Switch, create VLAN 400 and add ports "switch 1 CPU", Ether 4 & Ether 5 to it
Under Switch set Ether 2, Ether 3, Ether 4 & Ether 5 to VLAN mode = Secure and VLAN Header "always strip"
Under Switch set "Switch 1 CPU" to VLAN mode = Secure and VLAN Header "leave as is"
Under Switch set Ether 2 & Ether 3 Default VLAN ID to 200
Under Switch set Ether 4 & Ether 5 Default VLAN ID to 400

You can now add IP addresses, DHCP servers etc. to the VLAN interfaces and control routing between them using filters in the forward chain.

sashavl · Fri Feb 28, 2014 3:42 pm

Finnaly someone who understand switch chip vlan's. @dasiu Thank you.

dynek · Thu Mar 20, 2014 5:58 pm

You can check my MUM presentation about the switch chip: http://mum.mikrotik.com/presentations/I ... nowski.pdf

I read, re-read and re-re-read until I (think I) understood everything.

Based on your presentation, I now assume that VLAN interface(s) aren't required at all to manage VLANs if switch chip is being used, right ?

Thank you

efaden · Thu Mar 20, 2014 6:23 pm

You can check my MUM presentation about the switch chip: http://mum.mikrotik.com/presentations/I ... nowski.pdf
I read, re-read and re-re-read until I (think I) understood everything.

Based on your presentation, I now assume that VLAN interface(s) aren't required at all to manage VLANs if switch chip is being used, right ?

Thank you

That's correct as long as you don't want to have the routerboard see the traffic... When you use the switch chip VLAN interfaces serve generally to allow traffic from the routeros to the vlan.

dynek · Thu Mar 20, 2014 6:46 pm

So I can have switch chip declarations and beside this declare VLANs as interface as well ?

efaden · Thu Mar 20, 2014 6:48 pm

So I can have switch chip declarations and beside this declare VLANs as interface as well ?

Yeah... the switch chip controls which port vlans go to... the interface controls which vlans routeros actually sees for routing, ips, dhcp server, etc.

dasiu · Fri Mar 21, 2014 1:09 pm

You can check my MUM presentation about the switch chip: http://mum.mikrotik.com/presentations/I ... nowski.pdf
I read, re-read and re-re-read until I (think I) understood everything.

Based on your presentation, I now assume that VLAN interface(s) aren't required at all to manage VLANs if switch chip is being used, right ?

Thank you

Right. If you want tagged packets to travel from one port to another - you just use "/interface ethernet" to configure, which ports are controlled by switch, and then "/interface ethernet switch" to do the rest.
You use "/interface vlan" only if:
1. You want to add an IP address for the VLAN (for example - to route the traffic, to be able to ping hosts on the vlan from your MikroTik, etc.
2. If you want to bridge the vlan (the one on the switch) with other ports (wireless, tunnels, etc.)
3. If you want to sniff/torch something on the vlan (the switch should have then a mirroring or a rule with "copy to cpu").
Good to know, that the presentation actually helped someone. Thanks

dynek · Fri Mar 21, 2014 1:54 pm

I have been searching for an explanation for so long and couldn't understand how it all works.
So definitely your presentation is the best thing I've seen so far - Mikrotik's team should put it somewhere on the WiKi.

I even assume that some people think they are getting the best out of their router but they don't clearly know how to use switch chip for VLANs.

dynek · Fri Mar 21, 2014 9:36 pm

Dasiu, now between "I understood" and "I'm able to apply it", there's a world

In your presentation you mention only one master port for a chip. How if I want to have two but only one with switch chip used for vlans?
I currently have a RB450G (planning to switch for a 2011UiAS), can you tell me if my config looks correct ?
I am using ether1 for WAN, ether2 and ether3 to trunk VLAN, ether4 and ether5 with "non-vlan" ports for the moment:

ros code

/interface ethernet
set [ find default-name=ether1 ] mac-address=00:0C:42:BD:D3:F7 name=ether1-gateway
set [ find default-name=ether2 ] mac-address=00:0C:42:BD:D3:F8 name=ether2-master-trunk
set [ find default-name=ether3 ] mac-address=00:0C:42:BD:D3:F9 master-port=ether2-master-trunk name=ether3-slave-trunk
set [ find default-name=ether4 ] mac-address=00:0C:42:BD:D3:FA name=ether4-master-local
set [ find default-name=ether5 ] mac-address=00:0C:42:BD:D3:FB name=ether5-slave-local
/interface vlan
add interface=ether2-master-trunk l2mtu=1516 name=vlan100-management vlan-id=100
add interface=ether2-master-trunk l2mtu=1516 name=vlan200-private vlan-id=200
add interface=ether2-master-trunk l2mtu=1516 name=vlan300-guest vlan-id=300
/interface ethernet switch port
set 1 vlan-mode=secure
set 2 vlan-mode=secure
/interface ethernet switch vlan
add ports=switch1-cpu,ether2-master-trunk,ether3-slave-trunk switch=switch1 vlan-id=100
add ports=switch1-cpu,ether2-master-trunk,ether3-slave-trunk switch=switch1 vlan-id=200
add ports=switch1-cpu,ether2-master-trunk,ether3-slave-trunk switch=switch1 vlan-id=300

Thanks!

dcuk · Sat Mar 22, 2014 12:06 am

Unfortunately you can only have a single master port per switch chip. If you need to logically separate the ports you'll need to go with vlans. Of course, on an RB2011 there are two switch chips so you could wait for that.

dasiu · Sat Mar 22, 2014 11:10 am

Dasiu, now between "I understood" and "I'm able to apply it", there's a world

In your presentation you mention only one master port for a chip. How if I want to have two but only one with switch chip used for vlans?
I currently have a RB450G (planning to switch for a 2011UiAS), can you tell me if my config looks correct ?
I am using ether1 for WAN, ether2 and ether3 to trunk VLAN, ether4 and ether5 with "non-vlan" ports for the moment:

Did I mention, that the presentation is already on TikTube, ready to be watched? http://tiktube.com/video/CKhm3fCqjpGpDH ... sllJoKmmE=

As I said there - there can be only 1 master port in 1 switch chip... BUT, there is a trick

. If I understand it right, you need to switch ports ether4 and ether5 with no vlan tags, and they will be separated from the ether1-ether3 traffic? If it is so - then you can set ports ether4 and ether5 to have master ether2 (yes!), and be access ports in vlan 999 - let's say

. As only the 2 ports have vlan999, and they don't have any other vlan (no 100, 200, 300) and vlan mode is secure for all ports => ether1-ether3 trunk will be separated from ether4-ether5 switch. And vlan999 is not relevant, as they are both access - from the outside noone will see the tag, it's only internal.

efaden · Sat Mar 22, 2014 1:52 pm

Dasiu, now between "I understood" and "I'm able to apply it", there's a world

In your presentation you mention only one master port for a chip. How if I want to have two but only one with switch chip used for vlans?
I currently have a RB450G (planning to switch for a 2011UiAS), can you tell me if my config looks correct ?
I am using ether1 for WAN, ether2 and ether3 to trunk VLAN, ether4 and ether5 with "non-vlan" ports for the moment:
Did I mention, that the presentation is already on TikTube, ready to be watched? http://tiktube.com/video/CKhm3fCqjpGpDH ... sllJoKmmE=

As I said there - there can be only 1 master port in 1 switch chip... BUT, there is a trick . If I understand it right, you need to switch ports ether4 and ether5 with no vlan tags, and they will be separated from the ether1-ether3 traffic? If it is so - then you can set ports ether4 and ether5 to have master ether2 (yes!), and be access ports in vlan 999 - let's say . As only the 2 ports have vlan999, and they don't have any other vlan (no 100, 200, 300) and vlan mode is secure for all ports => ether1-ether3 trunk will be separated from ether4-ether5 switch. And vlan999 is not relevant, as they are both access - from the outside noone will see the tag, it's only internal.

That's clever.

Also note that on the crs you can actually have multiple master ports.

Sent from my SCH-I545 using Tapatalk

dynek · Sat Mar 22, 2014 9:10 pm

Thanks once more for your answer.

Still I am able to have two master ports in the configuration. Now, does it mean that if I do so I am not making use of switch chip to manage vlan traffic ?

efaden · Sat Mar 22, 2014 9:13 pm

Thanks once more for your answer.

Still I am able to have two master ports in the configuration. Now, does it mean that if I do so I am not making use of switch chip to manage vlan traffic ?

On what board?

The 2011 series has one switch chip for the GigE ports and one for the FastE ports. Any traffic that crosses between those two switch chips needs to go through the main processor. On the 2011 series switch chips the chip is capable of having only one master port per switch chip. If you try to configure more than one it will complain.

On the CRS you can have more than one... effectively what it does is that traffic between any slave port and a given master port is handled in the switch chip. Any traffic between master port groups would go through the main processor.

Is that what you were asking?

dynek · Sat Mar 22, 2014 9:20 pm

RB450G for the moment and RouterOS doesn't complain if I declare:
port 1 as WAN (gateway)

port 2 as master for my VLANs
port 3 as slave for my VLANs

port 4 as master for other IP range
port 5 as slave for other IP range

efaden · Sat Mar 22, 2014 9:27 pm

RB450G for the moment and RouterOS doesn't complain if I declare:
port 1 as WAN (gateway)

port 2 as master for my VLANs
port 3 as slave for my VLANs

port 4 as master for other IP range
port 5 as slave for other IP range

Don't have a 450G so I don't really know. Can't find the block diagram online. My assumption would be that if it lets you declare it then it should be able to handle it. You can email support just to verify.

dynek · Mon Mar 24, 2014 11:00 pm

OK so I finally went for one master port for WAN/NAT and one for VLANs, does this sound correct ?

[MODEM]----(ether1)[RB450G](ether2)----[SWITCH]----[NAS / Computer / ...]

However when Computer sends data to NAS, in current case I was synchronizing a 6Gb mailbox and then TimeMachine backup, CPU load of Mikrotik goes crazy, see attached file... I didn't expect this.

jkarras · Tue Mar 25, 2014 5:21 am

What is the bridge you have listed on there? That may be causing traffic to be sent to the CPU.

dynek · Tue Mar 25, 2014 9:47 am

That's a bridge between VLAN100 and MetaRouter's virtual interface.

I need it so my MetaRouter's instance gets an IP in the range of VLAN100.

jkarras · Tue Mar 25, 2014 9:24 pm

Anything flowing through the VLAN 100 is going to hit CPU then because of the bridge. Being a bridge with a metarouter interface will mean it can't go through fast path either.

As a test you could remove the bridge and do your file transfer.

dynek · Wed Mar 26, 2014 12:01 am

Would it work to disable the bridge or should I remove it ?
If so, I tried and it doesn't change - See screenshot.

jkarras · Wed Mar 26, 2014 1:33 am

Unknown on the remove vs disable question. Someone from Mikrotik can answer that.

The other thing I noticed that I should have noticed before. It looks like your systems are on different VLANs. So because its routing your going to hit CPU. Port to port traffic on the same VLAN would be switched by the switch chip. Traffic flowing from VLAN to VLAN must be routed so that means a CPU hit. Fast path can help the CPU hit but only if all the requirements are met.

http://wiki.mikrotik.com/wiki/Manual:Fast_Path

Do you have fast path turned on? Its listed under IP->Settings
Do you have any firewall rules enabled that would affect the routed traffic?

dynek · Wed Mar 26, 2014 1:51 am

Fast Path is not available on RB450G as far as I see / understand.

I haven't done anything regarding firewall rules for the moment, they are the default ones.

jkarras · Wed Mar 26, 2014 5:36 am

Makes sense then on the fast path. Based on the speed test results on the product page you are basically getting max speed for the RB450G. Assuming its a SMB share your transferring from its not surprising.

http://routerboard.com/rb450g

dynek · Wed Mar 26, 2014 9:39 am

Thank you for your answers. I didnt' think I would reach RB450G's limit that quick

Not a real problem though cause TimeMachine backups, etc. won't happen so often.

Chiverel · Fri May 18, 2018 6:34 pm

I'll dare to bump this old thread. I'm trying to understand vlans and essential topic seems to be just a right place.

There's a bunch of information on older ROS configuration. But I don't have solid knowledge for that and have problems in adjusting those configs into hew HW offload bridges/vlan/switch mixture.

My test device is RB2011 with ROS 6.42.1, reset with no default config. It has 2 independent switches:

CPU-integrated 100M
PCB mounted 1G switch that is connected to CPU with a 1Gb/s line

I'd appreciate essentials answers on the following topics:

Do I understand that bridge is implemented "above" switch. E.g. if switch is capable to deliver packet according to his ARP entries (Hosts tab) this packet doesn't even reach the Bridge interface? But when packet requires NAT-ing then it flows through bridge?
VLAN in Switch and Bridge menu. What are typical use cases? If I want to have vlan with DHCP and HW offloading, is it enough to configure it on the switch?
Example. I want to have Vlan id=10 on my ether4 and ether5 ports (both access with 2 PCs connected) with IP addresses assigned by router's DHCP. Here's what I have at the moment, but when I torch either bridge or ether4 directly and try to ping, I see empty column in VLAN ID

My test config

### create bridge
/interface bridge
add name=VLAN-Br protocol-mode=none

### set switch ports as Access mode ports, e.g. remove Vlan tag when packet leaves port and moves towards PC
### consider all untagged traffic from PC to port as Vlan-id=10
/interface ethernet switch port
set 4 default-vlan-id=10 vlan-header=always-strip vlan-mode=fallback
set 5 default-vlan-id=10 vlan-header=always-strip vlan-mode=fallback

### define ip pool for DHCP server
/ip pool
add name=LAN-pool ranges=192.168.10.10-192.168.10.30
### setup DHCP runnig on the Bridge
/ip dhcp-server
add add-arp=yes address-pool=LAN-pool disabled=no interface=VLAN-Br lease-time=1m name=LAN-dhcp

### add ports into Bridge (or I won't get IP addresses)
/interface bridge port
add bridge=VLAN-Br interface=ether4
add bridge=VLAN-Br interface=ether5

### group ether4 and ether5 into Vlan 10 on the switch chip
### if I remove switch1-cpu from ports, then I won't get DHCP, cause IP is already L3 which switch doesn't care about
/interface ethernet switch vlan
add independent-learning=no ports=ether4,ether5,switch1-cpu switch=switch1 vlan-id=10
/ip address
add address=192.168.10.1/27 interface=VLAN-Br network=192.168.10.0

With this config I connect 2 PCs and they can ping each other as well as the Bridge IP=192.168.10.1. Is this any close to a "vlan" or this is a complete crap? How do I see VLAN id values in the torch (example torch ether4, when ping is sent from PC to Bridge IP)?

Second test case. I have 2 "isolated" setups:

Bridge1G: ether1-5; switched VLAN=10 on ether1-5 + sw1_cpu, all Access ports; DHCP 192.168.10.0/23, pool 192.168.10.10-250
Bridge100M: ether6-10; switched VLAN=20 on ether6-10 + sw2_cpu, all Access ports; DHCP 192.168.20.0/23, pool 192.168.20.10-250

By linking any 2 ports from different bridges (let it be ether5 and ether6) with a patch cord. All my hosts connected to any vlan would be able to talk with each other. Because when I try to ping from ether1 to ether10:

packet from PC arrives untagged on ether1
packet receives VLAN=10
packet arrives tagged VLAN=10 on ether5
packet leaves ether5 without VLAN tag, because it is an access port and egress VLANs are removed
packet arrives untagged on ether6
packet receives VLAN=20
packet exits ether 10 without VLAN just like ether5

Is there anything wrong with this concept?

And the last one. Let's imagine:

I have a named Bridge-1G with DHCP with ether3-5
VLAN 10 are is assigned on a switch VLANs just like above (ether3-5, sw1-cpu)

I decide to extend my VLAN=10 on the switch2. So my actions are either:

Add Bridge-100M without DHCP, add ether6-ether10; Add same ports + sw2-cpu on a switch with vlan id=10. This would allow single DHCP to serve all ports, but traffic between ether groups 3-5 and 6-10 would go via CPU
Add vlan=10 with ether6-10 on a switch2; use a patch cord and connect ether5 and ether6. By doing this I loose 2 ports but don't use CPU when extending my 1G switch with 4x100M ports

Would A or B work? I mean the concept is somewhere close to correct?

Chiverel · Sat May 19, 2018 11:57 pm

And the last one. Let's imagine:

I have a named Bridge-1G with DHCP with ether3-5

VLAN 10 are is assigned on a switch VLANs just like above (ether3-5, sw1-cpu)

I decide to extend my VLAN=10 on the switch2. So my actions are either:

Add Bridge-100M without DHCP, add ether6-ether10; Add same ports + sw2-cpu on a switch with vlan id=10. This would allow single DHCP to serve all ports, but traffic between ether groups 3-5 and 6-10 would go via CPU

Add vlan=10 with ether6-10 on a switch2; use a patch cord and connect ether5 and ether6. By doing this I loose 2 ports but don't use CPU when extending my 1G switch with 4x100M ports
Would A or B work? I mean the concept is somewhere close to correct?

Option A doesn't work as described. 2nd bridge for ports 6-10 is not required. Ports should be added into 1st bridge Home-Br. Traffic between groups flows via CPU. I haven't tested how much can CPU process and what is a total bandwidth between switches.

Option B To exclude CPU from the flow, we need to use a patch cord to connect any port from the first and the second switch groups (then we have max of sfp + 4 x 1G + 4 x 100M switching without CPU), then apply following config:
- Create bridge "Home-1G" with ports: sfp, ether 1-5
- Add DHCP server on that bridge
- Create another bridge "Home-100M" with ports 6-10
- in the switch1 group ports: sfp1, ether1-5, sw1-cpu
- in the switch2 group ports: ether6-10
Example config is below (VLAN part is most likely a full trash according to Torch), connected ports are 4-7. The total bandwidth between switches is limited to 100Mbps.

# jan/02/1970 06:36:13 by RouterOS 6.42.1
# software id = 8NTX-HB0D
#
# model = 2011UiAS-2HnD
/interface bridge
add fast-forward=no name=100M-Br protocol-mode=none
add fast-forward=no name=Home-Br protocol-mode=none

/interface vlan
add interface=100M-Br name=100M-vlan20 vlan-id=20
add interface=Home-Br name=Home-vlan10 vlan-id=10

/interface ethernet switch port
set 4 default-vlan-id=10 vlan-header=always-strip vlan-mode=secure
set 5 default-vlan-id=10 vlan-header=always-strip vlan-mode=secure
set 6 default-vlan-id=20 vlan-header=always-strip vlan-mode=secure
set 7 default-vlan-id=20 vlan-header=always-strip vlan-mode=secure
set 11 default-vlan-id=10 vlan-header=add-if-missing vlan-mode=secure
set 12 default-vlan-id=20 vlan-header=always-strip vlan-mode=secure

/ip pool
add name=dhcp_pool0 ranges=192.168.10.10-192.168.10.30
/ip dhcp-server
add address-pool=dhcp_pool0 disabled=no interface=Home-Br lease-time=5m name=dhcp1

/interface bridge port
add bridge=Home-Br frame-types=admit-only-vlan-tagged interface=ether4 pvid=10
add bridge=Home-Br frame-types=admit-only-vlan-tagged interface=ether5 pvid=10
add bridge=Home-Br frame-types=admit-only-vlan-tagged interface=Home-vlan10 pvid=10
add bridge=100M-Br interface=ether6
add bridge=100M-Br interface=ether7
add bridge=100M-Br interface=100M-vlan20 pvid=20

/interface bridge vlan
add bridge=Home-Br tagged=Home-Br,ether4,ether5 vlan-ids=10
add bridge=100M-Br tagged=100M-Br,ether6,ether7 vlan-ids=20

/interface ethernet switch vlan
add independent-learning=no ports=ether4,ether5,switch1-cpu switch=switch1 vlan-id=10
add ports=ether6,ether7 switch=switch2 vlan-id=20

/ip address
add address=192.168.10.1/27 interface=Home-Br network=192.168.10.0

/ip dhcp-server network
add address=192.168.10.0/27 dns-server=192.168.10.1 gateway=192.168.10.1 netmask=27

I would appreciate clarifications on correct VLAN setup.

Chiverel · Mon May 21, 2018 8:10 pm

Maybe my experience would help someone, since topic is not really active. Summary of testing VLANs with HW offloading with the following config:

VLAN 10: access ports eth3, eth4; trunk eth5; DHCP 192.168.10.0/27
VLAN 20: access ports eth7, eth8; trunk eth6; DHCP 192.168.20.0/27

Here’s a picture (clickable)

Here’s the config after reset: bridge, ports, switch and DHCP. The rest is not used.

# create bridges for our Vlans
/interface bridge 
add name=Home-1G-br:vid10 protocol-mode=none fast-forward=no arp=proxy-arp pvid=10 vlan-filtering=no
add name=Home-100M-br:vid20 protocol-mode=none fast-forward=no arp=proxy-arp pvid=20 vlan-filtering=no   

# create Vlans with Bridges as parent interfaces. This would mark bridge traffic with a vlan tag
/interface vlan 
add name=vid10 interface=Home-1G-br:vid10 vlan-id=10                 
add name=vid20 interface=Home-100M-br:vid20 vlan-id=20

# rename interfaces just for clarity
/interface ethernet 
set [find default-name=ether3 ] name=eth3:vid10.1G.access
set [find default-name=ether4 ] name=eth4:vid10.1G.access  
set [find default-name=ether5 ] name=eth5:vid10.1G.trunk   
set [find default-name=ether6 ] name=eth6:vid20.100M.trunk    
set [find default-name=ether7 ] name=eth7:vid20.100M.access      
set [find default-name=ether8 ] name=eth8:vid20.100M.access  

# assign default vlan ids to ports and allow only VLAN frames with offload
/interface bridge port
add bridge=Home-1G-br:vid10 interface=eth3:vid10.1G.access frame-types=admit-only-vlan-tagged pvid=10 hw=yes ingress-filtering=yes 
add bridge=Home-1G-br:vid10 interface=eth4:vid10.1G.access frame-types=admit-only-vlan-tagged pvid=10 hw=yes ingress-filtering=yes 
add bridge=Home-1G-br:vid10 interface=eth5:vid10.1G.trunk frame-types=admit-only-vlan-tagged pvid=10 hw=yes ingress-filtering=yes
add bridge=Home-100M-br:vid20 interface=eth6:vid20.100M.trunk frame-types=admit-only-vlan-tagged pvid=20 hw=yes ingress-filtering=yes      
add bridge=Home-100M-br:vid20 interface=eth7:vid20.100M.access frame-types=admit-only-vlan-tagged pvid=20 hw=yes ingress-filtering=yes      
add bridge=Home-100M-br:vid20 interface=eth8:vid20.100M.access frame-types=admit-only-vlan-tagged pvid=20 hw=yes ingress-filtering=yes

# add interfaces in bridge Vlan. Remember to add bridge itself as a tagged member
/interface bridge vlan
add bridge=Home-1G-br:vid10 vlan-ids=10 tagged=Home-1G-br:vid10,eth5:vid10.1G.trunk untagged=eth3:vid10.1G.access,eth4:vid10.1G.access 
add bridge=Home-100M-br:vid20 vlan-ids=20 tagged=Home-100M-br:vid20,eth6:vid20.100M.trunk untagged=eth7:vid20.100M.access,eth8:vid20.100M.access

# assign IP configuration to Vlans 10 and 20
/ip address
add address=192.168.10.1/27 netmask=255.255.255.224 network=192.168.10.0 interface=vid10
add address=192.168.20.1/27 netmask=255.255.255.224 network=192.168.20.0 interface=vid20

# addresses pools for Vlans
/ip pool
add name=vid10-pool ranges=192.168.10.10-192.168.10.30
add name=vid20-pool ranges=192.168.20.10-192.168.20.30

# setup DHCP networks
/ip dhcp-server network
add address=192.168.10.0/27 netmask=255.255.255.224 dns-server=192.168.10.1 gateway=192.168.10.1
add address=192.168.20.0/27 netmask=255.255.255.224 dns-server=192.168.20.1 gateway=192.168.20.1

# setup DHCP servers on both Vlans
/ip dhcp-server
add address-pool=vid10-pool lease-time=01:00:00 interface=vid10 name=vid10-dhcp disabled=no
add address-pool=vid20-pool lease-time=01:00:00 interface=vid20 name=vid20-dhcp disabled=no

# strip or keep Vlan headers on egress switch ports. 
# make switch cpu chips vlans-aware
/interface ethernet switch port
set [find name=eth3:vid10.1G.access] vlan-mode=secure vlan-header=always-strip default-vlan-id=10
set [find name=eth4:vid10.1G.access] vlan-mode=secure vlan-header=always-strip default-vlan-id=10 
set [find name=eth5:vid10.1G.trunk] vlan-mode=secure vlan-header=add-if-missing default-vlan-id=10
set [find name=eth6:vid20.100M.trunk] vlan-mode=secure vlan-header=add-if-missing default-vlan-id=20    
set [find name=eth7:vid20.100M.access] vlan-mode=secure vlan-header=always-strip default-vlan-id=20     
set [find name=eth8:vid20.100M.access] vlan-mode=secure vlan-header=always-strip default-vlan-id=20 
set [find name=switch1-cpu] vlan-mode=secure vlan-header=leave-as-is   
set [find name=switch2-cpu] vlan-mode=secure vlan-header=leave-as-is

# add vlans members on the Switch chips
# remember to include SwitchX-cpu to be able to use: Nat, routing, torch etc.
/interface switch ethernet vlan
add switch=switch1 ports=switch1-cpu,eth3:vid10.1G.access,eth4:vid10.1G.access,eth5:vid10.1G.trunk vlan-id=10
add switch=switch2 ports=switch2-cpu,eth6:vid20.100M.trunk,eth7:vid20.100M.access,eth8:vid20.100M.access vlan-id=20

With that setup my computers can talk to the different VLANs over CPU by default, connections inside same switch are offloaded. Tests:

Laptop Win10 1803
Laptop Win10 1709

ethernet is set as a “private” network
Vlan=10 network=192.168.10.1/27
Vlan=20 network=192.168.20.1/27

I was pinging IP addresses from the PC in the row to a PC or VLAN IP in a column. Here are the results from ping attempts (clickable).

Torch shows proper VLAN ids. To see vlan marks:

torch an interface that is connected to Vlan (ex. bridge, or ethernet), I wasn’t able to see marks on vlan itself
select “VLAN id” checkbox
start torch again

Here I’ve connected PC1 to ether5, PC2 to ether6 and checking VLANs on both ports and bridges. Ping is launched to all 4 IP addresses: 2 vlans, 2 PCs. (clickable)

Summary

Adding more ports to the VLANs is easy, just take corresponding ports as an example and check switch port assignments carefully to enable/disable HW offload
Only 1 VLAN inside each switch group can be offloaded according to documentation, thus choose wisely to be efficient in performance

Improvement. How to isolate VLANs
We can create IP addresses list and use them in Firewall - Raw.

/ip firewall address-list
add list=vid10 address=192.168.10.1-192.168.10.30
add list=vid20 address=192.168.20.1-192.168.20.30
/ip firewall raw
add action=drop chain=prerouting src-address-list=vid20 dst-address-list=!vid20
add action=drop chain=prerouting src-address-list=vid20 dst-address-list=!vid20

Improvement. Switching more ports from Switch1 and Switch2
We can offload more ports by:

Moving more ports into single vlan
Connecting ports from different Switch groups by a patch cord (in this case we lose 2 ports: 1G and 100M port), but increase amount of hw switched ports. It’s worth to mention that connection between switches is limited to 100M
We can remove CPU link from the second switch group. Not sure whether it saves CPU cycles

I’m considering using this, because:

ISP provides me 100M channel
CPU load when WAN and LAN are connected in the 1G switch group is lower for about 5%, while allowing a bit more bandwidth (approx. +5Mbps). The test is synthetic and not the best planned, but I’d consider that when planning my network

Let's adjust config

# delete vlan20 related configs
/ip dhcp-server remove [find name=vid20-dhcp]
/ip dhcp-server network remove [find gateway=192.168.20.1]
/ip address remove [find network=192.168.20.0]
/ip pool remove [find name=vid20-pool]
/interface vlan remove [find name=vid20]
/interface ethernet switch vlan remove [find vlan-id=20]
/ip firewall address-list remove [find list="vid20"]
/ip firewall raw remove [find chain=prerouting]

# update ports, bridge and switch configs
/interface ethernet switch port set [find default-vlan-id=20] default-vlan-id=10
/interface bridge set [find name="Home-100M-br:vid20"] name="Home100M-br:vid10" pvid=10
/interface ethernet
set [find name="eth6:vid20.100M.trunk"] name="eth6:vid10.100M.trunk"             
set [find name="eth7:vid20.100M.access"] name="eth7:vid10.100M.access"               
set [find name="eth8:vid20.100M.access"] name="eth8:vid10.100M.access" 
/interface bridge port set [find pvid=20] pvid=10
/interface ethernet switch vlan add vlan-id=10 ports=eth6:vid10.100M.trunk,eth7:vid10.100M.access,eth8:vid10.100M.access switch=switch2

This is it. Keep in mind that traffic doesn’t reach the CPU and 100M switch chip doesn’t support “copy to CPU” feature. Thus if you want to see something, use ports from switch1 group or the “uplink” port to the switch2.

6-switched ports setup.

Same vlan on both switch chips
4 access ports: eth3, 4, 7, 8
2 trunk ports: eth5 and 6 are connected with a patch cord
single DHCP server
you still have some more ports to use in both 1G and 100M switch for WAN balancing, management, guest networks etc. But those will work through CPU

Complete config just in case someone is interested:

# RouterOS 6.42.1
# model = 2011UiAS-2HnD
/interface bridge
add arp=proxy-arp fast-forward=no name=Home-1G-br:vid10 protocol-mode=none
add arp=proxy-arp fast-forward=no name=Home100M-br:vid10 protocol-mode=none
/interface ethernet
set [ find default-name=ether3 ] name=eth3:vid10.1G.access
set [ find default-name=ether4 ] name=eth4:vid10.1G.access
set [ find default-name=ether5 ] name=eth5:vid10.1G.trunk
set [ find default-name=ether6 ] name=eth6:vid10.100M.trunk
set [ find default-name=ether7 ] name=eth7:vid10.100M.access
set [ find default-name=ether8 ] name=eth8:vid10.100M.access
/interface vlan
add interface=Home-1G-br:vid10 name=vid10 vlan-id=10
/interface ethernet switch port
set 3 default-vlan-id=10 vlan-header=always-strip vlan-mode=secure
set 4 default-vlan-id=10 vlan-header=always-strip vlan-mode=secure
set 5 default-vlan-id=10 vlan-header=add-if-missing vlan-mode=secure
set 6 default-vlan-id=10 vlan-header=add-if-missing vlan-mode=secure
set 7 default-vlan-id=10 vlan-header=always-strip vlan-mode=secure
set 8 default-vlan-id=10 vlan-header=always-strip vlan-mode=secure
set 11 vlan-mode=secure
set 12 vlan-mode=secure
/ip pool
add name=vid10-pool ranges=192.168.10.10-192.168.10.30
/ip dhcp-server
add address-pool=vid10-pool disabled=no interface=vid10 lease-time=1h name=vid10-dhcp
/interface bridge port
add bridge=Home-1G-br:vid10 frame-types=admit-only-vlan-tagged ingress-filtering=yes interface=eth3:vid10.1G.access pvid=10
add bridge=Home-1G-br:vid10 frame-types=admit-only-vlan-tagged ingress-filtering=yes interface=eth4:vid10.1G.access pvid=10
add bridge=Home-1G-br:vid10 frame-types=admit-only-vlan-tagged ingress-filtering=yes interface=eth5:vid10.1G.trunk pvid=10
add bridge=Home100M-br:vid10 frame-types=admit-only-vlan-tagged ingress-filtering=yes interface=eth6:vid10.100M.trunk pvid=10
add bridge=Home100M-br:vid10 frame-types=admit-only-vlan-tagged ingress-filtering=yes interface=eth7:vid10.100M.access pvid=10
add bridge=Home100M-br:vid10 frame-types=admit-only-vlan-tagged ingress-filtering=yes interface=eth8:vid10.100M.access pvid=10
/interface bridge vlan
add bridge=Home-1G-br:vid10 tagged=Home-1G-br:vid10,eth5:vid10.1G.trunk \
    untagged=eth3:vid10.1G.access,eth4:vid10.1G.access vlan-ids=10
add bridge=Home100M-br:vid10 tagged=Home100M-br:vid10,eth6:vid10.100M.trunk \
    untagged=eth7:vid10.100M.access,eth8:vid10.100M.access vlan-ids=20
/interface ethernet switch rule
add copy-to-cpu=yes ports=eth5:vid10.1G.trunk switch=switch1
/interface ethernet switch vlan
add independent-learning=yes ports=\
    switch1-cpu,eth3:vid10.1G.access,eth4:vid10.1G.access,eth5:vid10.1G.trunk switch=switch1 vlan-id=10
add ports=eth6:vid10.100M.trunk,eth7:vid10.100M.access,eth8:vid10.100M.access switch=switch2 vlan-id=10
/ip address
add address=192.168.10.1/27 interface=vid10 network=192.168.10.0
/ip dhcp-server network
add address=192.168.10.0/27 dns-server=192.168.10.1 gateway=192.168.10.1 netmask=27

Thank you for reading.

CZFan · Mon May 21, 2018 10:17 pm

@Chiverel, I have been following your last couple of posts in this topic/thread, must commend you for going at it and not just throw your question / problem over the fence, in true spirit of this forum I believe.

However, there are a couple of points I want to raise:
1. You must have a single bridge across all ports, if not, HW offload will be disabled on the 2nd, 3rd, etc.
2. I "personally" think that by using a patch cable, you might be limiting the speed between these switch groups to copper speed instead of using the internal backplane, also, in your current config, once traffic crosses from vlan10 to vlan20, you will go via cpu in any case.
3. I am not sure why you selected proxy-arp on the bridges, but that can cause issues and my suggestion will be to change this to "enabled"

Chiverel · Mon May 21, 2018 11:38 pm

@CZFan,
Thanks for your comments.

1. If you plan to Switch all ports, then yes. Since I'm planning to use eth2 as WAN, eth9 as Management and eth1+eth10 as reserved so far, and those ports won't be a part of a Home bridge. I don't see the point of enabling HW offload there. With the current setup I can see that HW offload is enabled on all 6 ports in 2 bridges (I'm running the last config)

 
/interface bridge port print 
Flags: X - disabled, I - inactive, D - dynamic, H - hw-offload 
 #     INTERFACE      BRIDGE         HW  PVID PR  PATH-COST INTERNA...    HORIZON
 0 I H eth3:vid10.... Home-1G-br:... yes   10 0x         10         10       none
 1   H eth4:vid10.... Home-1G-br:... yes   10 0x         10         10       none
 2 I H eth5:vid10.... Home-1G-br:... yes   10 0x         10         10       none
 3 I H eth6:vid10.... Home100M-br... yes   10 0x         10         10       none
 4 I H eth7:vid10.... Home100M-br... yes   10 0x         10         10       none
 5 I H eth8:vid10.... Home100M-br... yes   10 0x         10         10       none

2. Correct note that Vlan10 and 20 are bridged via CPU in the 1st scenario.

And the note regarding 100M on a patch cable as well. But in some cases it could be ok I guess. Imagine you have a programmable door lock / card reader, or a private web-server for your portfolio or whatever else that doesn't require fast connections and heavy data transfer to the other resources or internet, or asymmetric active/passive interface bonding. Than you can save some 1G ports and some CPU cycles. Practical usage is questionable, but it is rather test scenarios and attempts to reach what I have thought in theory.

I see that my "production" config causes CPU overloads sometimes and try to get as much as I can to reduce that load. What I was testing before. I create 2 separate bridges. Without any firewall, nat, mangle rules. Then I launch a 100M bandwidth test with iperf3 for 100 seconds:

1G-1G: avg.99.9Mbps with max 16% CPU peaks
1G-100M or 100M-1G: avg 94.7Mbps with CPU peaks at 22%
100M-100M: avg 94.7Mpbs @ max 21% CPU

With applying "patch cord" schema same 100M-1G / 1G-100M test causes peaks with humble 4% CPU. Thus is you have "slow" devices you get extra CPU cycles available. Of course benefits do not come for free. Anyway 10% CPU load reduction is awesome. The real-world scenarios would narrow the gap I guess, but I hope you got the idea.

3. Most likely proxy-arp is not required in current setup. But when I added VPN connections into the bridge I think it was exactly this option that allowed devices to talk to each other. I do not use this setting on bridges that are guest or unsecure.

However I'm just studying RB and networking at all. My previous experience was obtained on much simpler devices. I'm reviewing configurations I made in the beginning, running some tests. Corrections and explanations are welcome.

CZFan · Tue May 22, 2018 12:42 am

@CZFan,
Thanks for your comments.

1. If you plan to Switch all ports, then yes. Since I'm planning to use eth2 as WAN, eth9 as Management and eth1+eth10 as reserved so far, and those ports won't be a part of a Home bridge. I don't see the point of enabling HW offload there. With the current setup I can see that HW offload is enabled on all 6 ports in 2 bridges (I'm running the last config)
Code: Select all
 
/interface bridge port print 
Flags: X - disabled, I - inactive, D - dynamic, H - hw-offload 
 #     INTERFACE      BRIDGE         HW  PVID PR  PATH-COST INTERNA...    HORIZON
 0 I H eth3:vid10.... Home-1G-br:... yes   10 0x         10         10       none
 1   H eth4:vid10.... Home-1G-br:... yes   10 0x         10         10       none
 2 I H eth5:vid10.... Home-1G-br:... yes   10 0x         10         10       none
 3 I H eth6:vid10.... Home100M-br... yes   10 0x         10         10       none
 4 I H eth7:vid10.... Home100M-br... yes   10 0x         10         10       none
 5 I H eth8:vid10.... Home100M-br... yes   10 0x         10         10       none
...

Ahhh, apologies, I stand corrected, just tested again on my 2011, seems you can have multiple bridges on same device, when you have multiple switch chips, i.e. bridge per switch chip and HW offload will still be enabled, but you can't have multiple bridges per switch chip, the HW offload will be disabled on the 2nd, 3rd, etc bridge