Community discussions

MikroTik App
 
zespri
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 53
Joined: Sat Mar 26, 2016 1:45 pm

How to route between a bridge and a subnet?

Tue Sep 18, 2018 1:21 am

Hello all,

I have a hAP ac Lite that I configured like a switch:

I create a bridge, and I added ether2-ether5 to it selecting hw offload.

This "switch" is connected via, say either4, but as I understand it should not matter since I can plug in "root" or uplink anywhere.
So ether4 then is connected to a tp-link dumb switch which is in turn connected to another mikrotik, hAP ac, which is connected to my provider as pppoe client.This one has default 192.168.88.0/24 network, with dhcp, dns and what not.

I want to connect ether1 on my hAP ac Lite to a virtual switch that virtual machines will be connected to. I want them to be on a separate network, say 192.168.89.0/24. The virtual machines will have static ip addresses and use 8.8.8.8 as DNS so I do not really need DHCP and DNS for this.

So I gave my ether1 address 192.168.89.1 and on a virtual machine I set static address 192.168.89.2, gateway 192.168.89.1 and DNS 8.8.8.8.

I think part of configuration is missing because I cannot ping 192.168.88.1 from the VM and I cannot resolve dns or access internet at all.

I think some routing rules are missing on my hAP ac Lite to route between the bridge and ether1.

My goal here is to have the virtual machines on a separate subnet so they cannot access devices on the main subnet 192.168.88.2-255 but at the same time they need internet access via 192.168.88.1 and then my ISP. I'm also planning exposing some ports on these VMs so that an external connection can get through to them: ISP -> hAP ac -> tp-link -> hAP ac lite -> virtual router -> VM

I may be setting it up completely wrong.

How do I set them up? Thank you in advance.
PS. As is probably obvious by now I'm not very experience with networking, and I never worked with cisco or anything else, just Mikrotik, and just at home. I can do my research most of the time I just feel I'm missing some fundamental understanding how routing in Mikrotik works.

PPS it was pointed to me elsewhere that smart switch - > dumb switch -> smart switch configuration is not going to work. That's alright, I can plug my hAP ac Lite straight into hAP ac, without a dumb switch in-between, if it's gonna help.
 
mkx
Forum Guru
Forum Guru
Posts: 4634
Joined: Thu Mar 03, 2016 10:23 pm

Re: How to route between a bridge and a subnet?

Tue Sep 18, 2018 9:23 am

Beware that default configuration of "consumer-class" mikrotiks is to have ether1 configured as WAN port. Firewall rules are set accordingly. In your case this is quite wrong and you need to remove all firewall rules and set them according to your needs. In addition to that you either need to NAT IP addresses from the "virtual machines subnet - 192.168.89.x" on hAP ac lite or add route towards said subnet on your main router (using hAP ac lite as gateway).

Regarding disabling access to subnet 192.168.88.x from the virtual machines: you can get there by setting firewall rule on hAP ac lite which prevents communicating between "virtual server" subnet and most of "regular" subnet. But: it is not entirely fail safe as your main router could be used as gateway to access your "regular" LAN subnet.

The safe - and easier overall - way would be to use separate VLAN between both mikrotiks (I guess the dumb switch just might transparently pass it if it supports ethernet frames with size of 1508 bytes, most newer switches do). Then you would configure all routing and firewalling on your main router ... prevention of communication between "main" and "virtual server" subnet could be done 100% safe, you'd have single point of configuration etc. The hAP ac lite would again act only as switch (smart in this case). The link between both mikrotiks would become hybrid (tagged for virtual servers and untagged for the rest of LAN).
BR,
Metod
 
zespri
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 53
Joined: Sat Mar 26, 2016 1:45 pm

Re: How to route between a bridge and a subnet?

Tue Sep 18, 2018 10:08 am

Beware that default configuration of "consumer-class" mikrotiks is to have ether1 configured as WAN port. Firewall rules are set accordingly. In your case this is quite wrong and you need to remove all firewall rules and set them according to your needs.

I did reset the router before attempting this setup and I choose the option of not applying the default rules on prompt. In the end this is what I have:

# sep/15/2018 09:44:43 by RouterOS 6.43
# model = RouterBOARD 952Ui-5ac2nD
/interface bridge
add fast-forward=no name=bridge1
/interface ethernet
set [ find default-name=ether1 ] advertise=10M-half,10M-full,100M-half,100M-full,1000M-half,1000M-full
set [ find default-name=ether2 ] advertise=10M-half,10M-full,100M-half,100M-full,1000M-half,1000M-full
set [ find default-name=ether3 ] advertise=10M-half,10M-full,100M-half,100M-full,1000M-half,1000M-full
set [ find default-name=ether4 ] advertise=10M-half,10M-full,100M-half,100M-full,1000M-half,1000M-full
set [ find default-name=ether5 ] advertise=10M-half,10M-full,100M-half,100M-full,1000M-half,1000M-full
/interface wireless
set [ find default-name=wlan1 ] ssid=MikroTik
set [ find default-name=wlan2 ] ssid=MikroTik
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip hotspot profile
set [ find default=yes ] html-directory=flash/hotspot
/interface bridge port
add bridge=bridge1 interface=ether2 trusted=yes
add bridge=bridge1 interface=ether3 trusted=yes
add bridge=bridge1 interface=ether4 trusted=yes
add bridge=bridge1 interface=ether5 trusted=yes
/ip address
add address=192.168.89.1/24 interface=ether1 network=192.168.89.0
add address=192.168.88.99/24 interface=bridge1 network=192.168.88.0

In addition to that you either need to NAT IP addresses from the "virtual machines subnet - 192.168.89.x" on hAP ac lite or add route towards said subnet on your main router (using hAP ac lite as gateway).

Thank you for this I made some progress. This is what I added on the other router:

/ip route add distance=1 dst-address=192.168.89.0/24 gateway=192.168.88.99

I seem to be able to connect from 88 to 89 now. However I'm getting quite a lot of packets with state "invalid" in the log. With origin from .88 and destination on .89.
I cannot ping or access .88 computers from .89 (don't get replies) or internet (no route available).

If I remove this rule on the main router:

/ip firewall filter add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid

I can now access from .89 to .88 but I'm not sure why these are marked as invalid in the first place. And I don't know how to make internet accessible to .89.

Regarding disabling access to subnet 192.168.88.x from the virtual machines: you can get there by setting firewall rule on hAP ac lite which prevents communicating between "virtual server" subnet and most of "regular" subnet. But: it is not entirely fail safe as your main router could be used as gateway to access your "regular" LAN subnet.

I'll have to address that when I get the connection working, so I will revisit it at that time. I have a feeling that deny access on the firewall is going to be easier then setting up the correct routes.

The safe - and easier overall - way would be to use separate VLAN between both mikrotiks (I guess the dumb switch just might transparently pass it if it supports ethernet frames with size of 1508 bytes, most newer switches do). Then you would configure all routing and firewalling on your main router ... prevention of communication between "main" and "virtual server" subnet could be done 100% safe, you'd have single point of configuration etc. The hAP ac lite would again act only as switch (smart in this case). The link between both mikrotiks would become hybrid (tagged for virtual servers and untagged for the rest of LAN).

The dumb switch is not listed by manufactorer as vlan capable (tp-link TL-SG1008D) so I was hesitant to go down VLAN road because I would not know if it does not work because of the dumb switch or because of me doing it wrong.

Thank you for your response. If you could help further on the points above I would very much appreciate it.
 
mkx
Forum Guru
Forum Guru
Posts: 4634
Joined: Thu Mar 03, 2016 10:23 pm

Re: How to route between a bridge and a subnet?

Tue Sep 18, 2018 11:11 am

If the configuration you posted is complete, then at least default route is missing ... without it, devices in 192.168.89.x won't have internet access:
/ip route
add gateway=192.168.88.1
One reason you're getting invalids on the main router is that devices in 192.168.88.x don't know how to access 192.168.89.x ... so they will send their traffic to their default gateway (192.168.88.1). That one will pass packets to your hAP ac lite (192.168.88.99) and that one will send them directly to target (virtual) machine. Replies will be sent by (virtual) machine to their default gateway (hAP ac lite at 192.168.89.1), which in turn knows how to deliver them directly to target machines (as it has route towards 192.168.88.x subnet directly). So main router will never see replies and it's connection tracking state will be fscked-up.
If you're going to allow all traffic between both subnets, then you can instruct firewall on main router not to connection-track these connections by adding a raw firewall rule:
/ip firewall raw
add chain=prerouting action=notrack src-address=192.168.88.0/24 dst-address=192.168.89.0/24
# I don't think you need another rule for the opposite direction. If you needed it, connection tracking would be fine in the first place
.
If you want to limit connection between both subnets, then you will have to tackle it differently. In this case the easiest way would be to deploy VLANs. If not using VLANs, you'll have to use firewall on hAP ac lite, that's the box that can do proper connection tracking of connections between both subnets.

Regarding VLANs: dumb switches normally don't explicitly support VLANs (they have to be managed for that obviously), but supporting "mini jumbo" frames with size 1508 is another thing (might be advertised or not). So you might want to try. OTOH, you mentioned you could connect both RBs directly bypassing the said switch. Personally I'd go that way.
BR,
Metod
 
zespri
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 53
Joined: Sat Mar 26, 2016 1:45 pm

Re: How to route between a bridge and a subnet?

Tue Sep 18, 2018 11:51 am

I cannot thank you enough, this explanation makes total sense. After adding 192.168.88.1 as the gateway on the hAP ac lite the virtual machines got internet access. Also thank you very much for taking time to explain why connection tracking gets confused, this is very educational. I'll try to google how to set up VLANs properly and I'll come back if I have questions.

The second reason I'm reluctant with VLANs is that they need to be setup on the main (hAP ac) router and it means that the household will be without internet while I'm monkeying arond with the VLANs. Trying to get firewall rules on hAP ac Lite at the moment sounds more preferable to me. I do want to isolate the networks.


You wrote

OTOH, you mentioned you could connect both RBs directly bypassing the said switch. Personally I'd go that way

I still need the dumb switch to connect the rest of the network; Is it possible to use VLAN only for one subnet and not use it for the other?
 
mkx
Forum Guru
Forum Guru
Posts: 4634
Joined: Thu Mar 03, 2016 10:23 pm

Re: How to route between a bridge and a subnet?

Tue Sep 18, 2018 2:56 pm

OTOH, you mentioned you could connect both RBs directly bypassing the said switch. Personally I'd go that way
I still need the dumb switch to connect the rest of the network; Is it possible to use VLAN only for one subnet and not use it for the other?
It surely is possible. As VLANs are a layer just above physical ethernet and below IP layer, and are add-on to "normal" ethernet, you can run mixed network ... some device connections over VLAN and other over "normal" ethernet. Further more, you can have both (VLAN tagged and untagged, i.e. "normal") running over same physical wire / set of ethernet ports. Thus, if your dumb switch does support "baby jumbo" frames, you can mix vlan-tagged (between both routerboards) and untagged (between the rest of LAN devices) traffic on the same wire. If it does not, then you will connect the routerboards directly (and you'll configure tagged VLANs only on those two ethernet ports), the dumb ethernet switch will be connected to one of "access" (i.e. VLAN untagged) ports of either main router or hAP ac lite, whichever suites your situation better.

When you get to configure routerboards, there are a few settings to be done:
  • first you need to configure selected ethernet ports with allowed set of VLAN IDs.
  • then you need to add "vlan interfaces" ... these are special devices that will allow you to configure certain functionality of routerboard to select VLANs. When those "vlan interfaces" are created, you configure stuff just as on any other interfaces (ethernet, wlan, ...). E.g. you will set separate IP address to that device (and routerboard will be accessible through this IP address from devices that will belong to selected VLAN), you will probably run DNS server on vlan device. You could have run DHCP server as well, but you mentioned your virtual servers all have addresses set statically.
  • there are a few details to be done for current (i.e. untagged) network to keep it working after the next step
  • at the end, you enable vlan-filtering on the bridge, so that bridge will kind of transform from being "dumb switch" to being "smart switch".
You will have to do similar VLAN-related setup on both routerboards. Afterwards hAP ac Lite will act more or less as smart switch (again), all the routing and firewalling (now for two LAN segments) will be done by main router.

[edit] Regarding separation of both networks without VLANs: even if you deploy firewalling on hAP ac lite, you still need to "untrack" those connections on the main router. Just because you control connections on hAP ac lite it doesn't help main router to track connection any better.
You will have to be careful when you'll construct firewall filters on hAP ac lite. You'll probably want to drop connections from main LAN subnet (IP addresses 192.168.88.0/24) towards virtual serves and vice versa, but you might want (or not, I'm not sure) allow at least part (if not full) connectivity to and from default gateway. I imagine default gateway might send some ICMP data (e.g. fragmentation needed or some such) from time to time and it wouldn't be good if those would go missing.

[edit2] I checked specifications of your dumb switch. Specifications say that it supports proper jumbo frames (up to 15Kb) so it should pass VLAN tagged packets just fine.
What VLAN-capable switch does and your dumb switch doesn't do is to keep broadcast domains separate between VLANs. When an ethernet device has to pass a frame to another ethernet device, it needs to learn MAC address of destination. So it performs ARP discovery procedure. After that it send ethernet frame to destination MAC address. Switches learn MAC addresses of devices, connected to individual ethernet ports, so that they can send unicast ethernet frames only via correct ethernet ports (if MAC address is yet unknown, they send such frame to all ethernet ports, the same happens with multicast/broadcast ethernet frames). If a switch is VLAN capable, it will learn (and remember) MAC addresses also per VLAN while your switch will not care about VLANs.
Another thing that VLAN-capable switch does is that it actually filters traffic according to VLAN tags. If one port receives traffic with VLAN which is not allowed on that port, it will drop the frame. And it will not send out a frame (not even a broadcast/multicast frame) via port that doesn't have appropriate VLAN configured. Dumb switch will not care about either.

VLAN being also a security thing ... in your case your dumb switch will leak traffic to wrong ports. But as it's your home setup you probably don't care about it too much. Just be careful when those neighbour kids pay a visit :wink:
BR,
Metod
 
zespri
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 53
Joined: Sat Mar 26, 2016 1:45 pm

Re: How to route between a bridge and a subnet?

Wed Sep 19, 2018 5:01 am

Thank you this is most helpful. There is quite a lot to take in, but while I'm doing that:
/ip firewall filter
add action=drop chain=forward connection-state=new dst-address=192.168.88.0/24 in-interface=ether1
From my newbie perspective this line on the hAP ac Lite should achieve the desired separation. I will still be able to access boxes on 192.168.89 from my main network (.88) and the VMs has no way of initiating connections to .88.

This does not look very complicated. What am I missing?

Thank you!
 
mkx
Forum Guru
Forum Guru
Posts: 4634
Joined: Thu Mar 03, 2016 10:23 pm

Re: How to route between a bridge and a subnet?

Wed Sep 19, 2018 9:17 am

The rule you posted will do most of work. It doesn't prevent from establishing untracked connections, such as most UDP connections (unless that's handled by some app helper that understands the application behaviour, I don't know if ROS has some). You can verify this by running iperf in UDP mode (run in server mode on some LAN machine and in client mode on one of servers).

It doesn't plug all the holes though. More paranoid firewall administrators will have a few firewall rules which explicitly allow certain connections and the last one will be rule that categorically drops anything. So your rule might be transcribed roughly to
/ip firewall filter
add action=allow chain=forward connection-state=established,related src-address=192.168.89.0/24 dst-address=192.168.88.0/24 comment="Allow traffic from servers to LAN only when connection is established from LAN"
add action=allow chain=forward src-address=192.168.88.0/24 dst-address=192.168.89.0/24 protocol=tcp dst-port=22 comment="Allow SSH connections from LAN to servers"
# add some other explicitly allowed connections, such as HTTP/HTTPS, perhaps SMTP, SMB, ICMP, who knows?
add action=drop chain=forward src-address=192.168.88.0/24 dst-address=192.168.89.0/24 comment="drop anything else between two subnets (LAN->servers)"
add action=drop chain=forward src-address=192.168.89.0/24 dst-address=192.168.88.0/24 comment="drop anything else between two subnets (servers->LAN)"
# you might want to filter some connectivity from servers towards internet as well. This might limit damage extent in case some server gets compromised if hack needs additional components from internet and your setup prevents getting them
.
I'm not saying you're on the paranoid side, neither am I. :wink:
BR,
Metod
 
Bobstonom
just joined
Posts: 7
Joined: Tue Sep 11, 2018 6:17 pm
Location: Tampa, Florida

Re: How to route between a bridge and a subnet?

Wed Sep 19, 2018 9:51 am

"It doesn't plug all the holes though. More paranoid firewall administrators will have a few firewall rules which explicitly allow certain connections and the last one will be rule that categorically drops anything."
Hmm, good point. :)
 
zespri
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 53
Joined: Sat Mar 26, 2016 1:45 pm

Re: How to route between a bridge and a subnet?

Sat Jun 20, 2020 2:00 am

@mkx Thank you again, for your explanation about VLANs. I have some hardware changes, and it looks like it's time for me to bite the bullet and configure the VLANs.

I have this question though: what is the advantage of the VLAN in my scenario (it's largely unchanged with a few minor changes)? What does VLAN allows to do that is tricky with simply different subnets as we/I tried before (as above) and why?
 
zespri
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 53
Joined: Sat Mar 26, 2016 1:45 pm

Re: How to route between a bridge and a subnet?

Sat Jun 20, 2020 3:10 am

I'm sorry, my drawing skills suck, the diagram below reflects what has been discussed above:

Image

Here Mikrotik hAP ac have a wireless network that most of the devices are connected to, this is on 192.168.88.* as described above in the thread.
The patch panel carries Ethernet cables around the house, and this is how wired connection for many devices are done.

Port 4 on Mikrotik hAP ac and Port 4 on Mikrotik hEX represent physical devices plugged in strait to router.

The powerage box used to be a homelab. It ran hypervisor used to have management and VMs separate ports, and another one was idrac port for power management. I do not have the powerage any longer but I'm going to replace it with a single Ethernet port workstation that will also run hypervisor. It's single Ethernet port will be shared by VMs and management.

I also have currently L2TP VPN setup on the Mikrotik hAP ac thanks to this thread.

What I want to achieve:

I want to isolate the hypervisor and VMs into a separate (192.168.89.*) network.
I want internet / DNS (which is on 192.168.88.1 now) be accessible from 192.168.89.* I do NOT want 192.168.88.* be normally accessible from 192.168.89.*.
However I want to be able to access 192.168.89.* from 192.168.88.* where I need to, and of course I need to have a way to forward particular ports open to internet on 192.168.88.1 to various 192.168.89.* VMs.
I have some special DNS requirements (which are working to my satisfaction right now) that all DNS queries (port 53) no matter what external IP they are sent to, are sent to IP that I have configured instead. I do not want to lose this property, and this should be true for both 192.168.88.* and 192.168.89.*.
/ip firewall nat
add action=redirect chain=dstnat dst-port=53 protocol=tcp
add action=redirect chain=dstnat dst-port=53 protocol=udp

The above was mostly working with my old setup (described in this thread), but I had a little bit less isolation that I wanted to (see discussion above re: connection-less access). I think it's time to do VLAN setup, if it can solve it.

I also want to have different VPN users, some of them have full access like right now, and some of them are limited by 192.168.89.* only. This way I can give access to someone to play with the hypervisor and VMs but not to the rest of my home network.

This post gives a general reference and context to what I'm doing. At the moment, as per my previous post, I'm trying to understand how and why VLAN is going to help me, that is why it can achieve more that the current setup without VLAN.

I'm guessing that I'll be aproaching setting up VLAN based on this:
When you get to configure routerboards, there are a few settings to be done:
  • first you need to configure selected ethernet ports with allowed set of VLAN IDs.
  • then you need to add "vlan interfaces" ... these are special devices that will allow you to configure certain functionality of routerboard to select VLANs. When those "vlan interfaces" are created, you configure stuff just as on any other interfaces (ethernet, wlan, ...). E.g. you will set separate IP address to that device (and routerboard will be accessible through this IP address from devices that will belong to selected VLAN), you will probably run DNS server on vlan device. You could have run DHCP server as well, but you mentioned your virtual servers all have addresses set statically.
  • there are a few details to be done for current (i.e. untagged) network to keep it working after the next step
  • at the end, you enable vlan-filtering on the bridge, so that bridge will kind of transform from being "dumb switch" to being "smart switch".
You will have to do similar VLAN-related setup on both routerboards. Afterwards hAP ac Lite will act more or less as smart switch (again), all the routing and firewalling (now for two LAN segments) will be done by main router.
I'm going to need to fill in some missing details here, but I'll do that one step a time once I understand the overall idea.
 
zespri
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 53
Joined: Sat Mar 26, 2016 1:45 pm

Re: How to route between a bridge and a subnet?

Sat Jun 20, 2020 3:16 am

One reason you're getting invalids on the main router is that devices in 192.168.88.x don't know how to access 192.168.89.x ... so they will send their traffic to their default gateway (192.168.88.1). That one will pass packets to your hAP ac lite (192.168.88.99) and that one will send them directly to target (virtual) machine. Replies will be sent by (virtual) machine to their default gateway (hAP ac lite at 192.168.89.1), which in turn knows how to deliver them directly to target machines (as it has route towards 192.168.88.x subnet directly). So main router will never see replies and it's connection tracking state will be fscked-up.
If you're going to allow all traffic between both subnets, then you can instruct firewall on main router not to connection-track these connections by adding a raw firewall rule:
So I tried to draw diagram above, and re-reading this here is a question I have. You wrote:
Replies will be sent by (virtual) machine to their default gateway (hAP ac lite at 192.168.89.1), which in turn knows how to deliver them directly to target machines (as it has route towards 192.168.88.x subnet directly). So main router will never see replies and it's connection tracking state will be fscked-up.
First of all, I'm assuming that " (hAP ac lite at 192.168.89.1)" should read " (hEX at 192.168.89.1)". If that's the case, then if you'll look at that diagram, you'll notice that hEX connected to the rest of the network via hAP ac lite only. I do not see how it could be true that the main router never see the packets, there is no other path for them to travel to other devices, other than via the main router? From the observations, everything you said is true, I would just like to understand how this particular aspect works.

Thank you in advance.
 
mkx
Forum Guru
Forum Guru
Posts: 4634
Joined: Thu Mar 03, 2016 10:23 pm

Re: How to route between a bridge and a subnet?

Sat Jun 20, 2020 12:45 pm

... if you'll look at that diagram, you'll notice that hEX connected to the rest of the network via hAP ac lite only.

Physically yes ... but logically doesn't have to be so, but that largely depends on configuration of hAP ac. Default config has ports ether2-ether5 bridged and bridge is L2 device, transparent for L3 (in this case that's IP). Which means that even though physically traffic between e.g. Shkaf and anything connected to TP-link passes hAP ac, IP layer (routing entity) of hAP ac won't see it. There are a few possibilities to force this traffic through hAP ac's IP layer (e.g. firewall), or to change config as I suggested two years ago.

I wonder what made you to resurrect this ancient thread?
BR,
Metod
 
zespri
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 53
Joined: Sat Mar 26, 2016 1:45 pm

Re: How to route between a bridge and a subnet?

Sat Jun 20, 2020 1:45 pm

I wonder what made you to resurrect this ancient thread?
I thought I explained it above ;) I guess not very well.

The set up that we discussed at the start of the thread and that I implemented with your help is what I've been running since 2018. The diagram above is also dated that time, and apart from the Powerage box both the setup and the diagram are still up to date.
I used to have Dell Powerage for running hypervisor, but I had to return it recently and I got myself Dell Precision instead. It will be a bit different from the networking perspective because Powerage had 3 ports and Precision has only one.

On the other hand the box I got now is quite a bit more powerful, so I intend to share it with a few friends who are interested. So I take this as an opportunity to improve security my setup and also learn a bit more about VLANs and how to set it up.

Rather starting a new thread and repeating all this again, I thought that it would be logical to continue where I left off.

I hope that explains it. I'm open to any suggestions, and if you feel I should open a new thread, I don't mind doing that, I just did not see a value in it myself.

Thank you for your answer.
or to change config as I suggested two years ago.
If I understand correctly to what change you are referring to, it's to configure VLAN. And this is exactly the route I'd like to take right now. Back then it seemed a bit overwhelming, but I got comfortable with the current set up and feel more confident going from there.

I read an article a few months ago which said that VLAN is just a way to create new broadcast domain, and that it's the same thing as using a physical switch, only it does not require cables and additional hardware because it's virtual. And that functionally whether you have two separate broadcast domains via VLAN or via subnets, does not matter.

This got me confused, because you told me that some limitation that we discussed back then can be overcome with VLAN, so it means that they are not exactly functionally equivalent. So I thought that as a first step of setting up VLAN, I could familiarise myself with why VLAN can achieve what we could not achieve otherwise. If you could explain that to me, that would be awesome.

On the other hand, if you think I'm looking at it entirely wrong way, I'm also happy to take your lead and start elsewhere. Thank you very much for responding again, I really appreciate your time and help.
 
mkx
Forum Guru
Forum Guru
Posts: 4634
Joined: Thu Mar 03, 2016 10:23 pm

Re: How to route between a bridge and a subnet?

Sat Jun 20, 2020 5:14 pm

Even though I quickly read past posts I must admit it's not clear to me how should logical topology of your network look like. As you're willing (and have opportunity) to change things, perhaps this is good opportunity to think of requirements (e.g. how many LAN segments do you want to have and what are drivers for each to have it separate from the rest). When you have that covered, engineer the physical solution (having in mind the equipment at hand). You should not consider existing topology at this time, only consider physical constraints if there are any (such as placement of client devices and max number of network cables available for connection different locations).
BR,
Metod
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 5129
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: How to route between a bridge and a subnet?

Sat Jun 20, 2020 5:34 pm

I am just enjoying mkx and vlans! :-)
You are in expert hands zespri!!
I'd rather manage rats than software. Follow my advice at your own risk! (Sob & mkx forced me to write that!)
MTUNA Certified, by the Ascerbic Llama!
 
zespri
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 53
Joined: Sat Mar 26, 2016 1:45 pm

Re: How to route between a bridge and a subnet?

Sun Jun 21, 2020 1:59 am

Even though I quickly read past posts I must admit it's not clear to me how should logical topology of your network look like. As you're willing (and have opportunity) to change things, perhaps this is good opportunity to think of requirements (e.g. how many LAN segments do you want to have and what are drivers for each to have it separate from the rest). When you have that covered, engineer the physical solution (having in mind the equipment at hand). You should not consider existing topology at this time, only consider physical constraints if there are any (such as placement of client devices and max number of network cables available for connection different locations).

Thank you for this. As far as requirements go, I believe I listed them a few posts above. Unfortunately, I'm not sure how to bridge the game from that to particular changes I need to do on device(s). I believe that the physical solution would not be much different from the existing one, since there is only so many ways to connect the few devices that I have. I believe that the physical design will remain mainly unchanged. Here is an updated diagram:
Image
  • Patch Panel - Located in the network cabinet, cables run to the rest of the house. All sockets should be part of the Home Network, apart from Dell Precision one, which will be used for both Hypervisor and VMs and will be on the VM network
  • TP-LINK TL-SG1008D - Located in the network cabinet, dumb switch for getting network connection from the router to the patch panel
  • Microtik hAP ac - Located in the network cabinet, the main router which is connected to the internet provider, and has WIFI capability to connect all household wireless device. The latter are part of the Home Network.
  • Microtik hEX - Located in the network cabinet, used to be used to create a separate network for VMs, but may be used as a simple switch. If it is desirable to get connection to the VM network through it (not shown on the diagram) one of it ports can be plugged into Patch Panel port 8. I'm not sure though if there are any advantages of that.
  • Internet Provider - an ONT Terminal located in the network cabinet
  • Fingbox - physical appliance, located in the network cabinet
  • shkaf - a small server, located in the network cabinet
  • alex2020-eth, a computer located elsewhere in the house, that requires a wired connection, and does not have a free socket next to it. Connected by Ethernet cable, run along the wall.
  • Dell Precision, the server for hosting hypervisor and VMs that should be on separate network. Located in a different room and connected via wall socket.
Goals:
  • Dell Precision lives on the VM LAN and everything live on the Home LAN
  • No direct access from VM LAN to Home LAN, but see below
  • DNS requests to any external DNS server IP have to end up on the Main router and resolved there. It includes VM LAN
  • IPs and Ports from the VM LAN need to be able to be exposed on the Home LAN and to the internet selectively
  • L2TP VPN needs to provide access either to both LANs or to just VM LAN based on what user is dialling in.

Now all of the above are not hard requirements, this is my wish-list, that I'd like to implement. The diagram and the connection all can be changed. I think, that most of the current state is usable as is but I do not mind disregarding it, if it gets on the way of how things are "supposed" to be, it's just there is probably no point to re-doing something from scratch if the end result will be quite similar, if that's the case, certain existing parts can be reused. Having said that, I do not mind changing things either. I also have some spare cables and I think I can get more if needed.

I would appreciate any help with setting this up. I'm especially vague on the VLANs part, as I'm not exactly sure how that works.

I am just enjoying mkx and vlans! :-)
You are in expert hands zespri!!
He already proved that times and again, I have no shred of doubt about it!
 
mkx
Forum Guru
Forum Guru
Posts: 4634
Joined: Thu Mar 03, 2016 10:23 pm

Re: How to route between a bridge and a subnet?

Sun Jun 21, 2020 12:30 pm

OK, here are my thoughts:

TL is unmanaged, so it'll be untagged part of network. hEX can be set up as managed switch, but since everything connected to it will be part of same LAN segment (Home LAN), it doesn't have to be set up with VLANs. The only device bound to be part of separate LAN segment is Dell, but it's going to be connected directly to main router (hAP ac).
IMHO with your current goals it's not worth to play with VLANs unless you want to get acquainted with it. It would be enough if you set up hAP ac as follows:
  • create bridge for Home LAN and add ether2-4 plus wlan
  • configure IP on ether1 (WAN), ether5 (VM LAN) and bridge (Home LAN)
  • set up firewall
Probably the best approach for the above would be to start off default config, which assumes ether1 for WAN snd bridges everything else for LAN, firewall filter rules are set appropriately. Then you remove ether5 from bridge and do whatever config. Unless you explicitly add ether5 to LAN interface list firewall will already disallow connections between VM and Home LAN. hEX should be configured as plain (dumb) switch.

If you decide to play with VLANs, then come back and we'll throw in some ideas about how to proceed. High-level (IP addresses, routes, furewall) set-up will be the same as the one without VLANs, only interfaces will be different (bridges on both Mikrotiks will span all interfaces, including ether1 and ether5, there will be at least 3 VLANs, some ports will be trunk while most will be access ports for one of VLANs ...).

BTW, which hEX generation exactly are we talking about? Depending on that it might happen that your hEX won't be able to HW offload intra-VLAN traffic so there might be some performance hit. hAP ac should be fine. OTOH it might be better suited for the role of main router ...
As long as you stick to non-VLAN setup both devices can HW-offload most of LAN-to-LAN trafffic.
BR,
Metod
 
zespri
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 53
Joined: Sat Mar 26, 2016 1:45 pm

Re: How to route between a bridge and a subnet?

Sun Jun 21, 2020 1:33 pm

First of all, thank you very much, most of this makes perfect sense to me.

TL is unmanaged, so it'll be untagged part of network.

Earlier in this thread you wrote:

[edit2] I checked specifications of your dumb switch. Specifications say that it supports proper jumbo frames (up to 15Kb) so it should pass VLAN tagged packets just fine.

I understand, that the proposal is not to use VLAN, but would it be possible to use VLAN with that in principle? This is not a very important question, but it's just to improve my understanding.

Secondly, someone else suggested that I should have multiple VLANs on hypervisor. One for management networks and another one or two for VMs depending on how I want to isolate them. This sounds appealing to me. I realise that I mentioned nothing like this above, and I don't want to look like a moving goal post, but I'm learning, and I do not always know what's possible. Now when I know that I have this option it looks like reasonable to me. I was told that hypervisor can act as a manageable switch from networking point of you and tag multiple VLANs. Would this be possible to support from the Mikrotik side?

TL is unmanaged, so it'll be untagged part of network. hEX can be set up as managed switch, but since everything connected to it will be part of same LAN segment (Home LAN), it doesn't have to be set up with VLANs. The only device bound to be part of separate LAN segment is Dell, but it's going to be connected directly to main router (hAP ac).
IMHO with your current goals it's not worth to play with VLANs unless you want to get acquainted with it. It would be enough if you set up hAP ac as follows:
  • create bridge for Home LAN and add ether2-4 plus wlan
  • configure IP on ether1 (WAN), ether5 (VM LAN) and bridge (Home LAN)
  • set up firewall
Probably the best approach for the above would be to start off default config, which assumes ether1 for WAN snd bridges everything else for LAN, firewall filter rules are set appropriately. Then you remove ether5 from bridge and do whatever config. Unless you explicitly add ether5 to LAN interface list firewall will already disallow connections between VM and Home LAN. hEX should be configured as plain (dumb) switch.

Understood. I'll give that a try.

If you decide to play with VLANs, then come back and we'll throw in some ideas about how to proceed. High-level (IP addresses, routes, furewall) set-up will be the same as the one without VLANs, only interfaces will be different (bridges on both Mikrotiks will span all interfaces, including ether1 and ether5, there will be at least 3 VLANs, some ports will be trunk while most will be access ports for one of VLANs ...).

Interesting. I have found Manual:Basic VLAN switching but it seems it only applies to hardware switches, where as I do not have one: both my Mikrotiks are routers.In particular they do not support `/interface ethernet switch egress-vlan-tag` command. So how do you set up a trunk work with a router?

Why do you think 3 VLANs would be required?

BTW, which hEX generation exactly are we talking about? Depending on that it might happen that your hEX won't be able to HW offload intra-VLAN traffic so there might be some performance hit. hAP ac should be fine. OTOH it might be better suited for the role of main router ...
As long as you stick to non-VLAN setup both devices can HW-offload most of LAN-to-LAN trafffic.

I'm not sure about the generation thing. How do I find out? This is a hEX. My one also says RB750Gr3 on the label. There are no word "generation" anywhere on the labels or I cannot find it. It gives a serial number and a MAC, and that's it. What am I looking for?

As always, thank you for you help!
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 5129
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: How to route between a bridge and a subnet?

Sun Jun 21, 2020 2:21 pm

This is the best reference for the newer bridge vlan way of doing things, if needed.
viewtopic.php?t=143620
I'd rather manage rats than software. Follow my advice at your own risk! (Sob & mkx forced me to write that!)
MTUNA Certified, by the Ascerbic Llama!
 
mkx
Forum Guru
Forum Guru
Posts: 4634
Joined: Thu Mar 03, 2016 10:23 pm

Re: How to route between a bridge and a subnet?

Sun Jun 21, 2020 3:05 pm

TL is unmanaged, so it'll be untagged part of network.

Earlier in this thread you wrote:

[edit2] I checked specifications of your dumb switch. Specifications say that it supports proper jumbo frames (up to 15Kb) so it should pass VLAN tagged packets just fine.

I understand, that the proposal is not to use VLAN, but would it be possible to use VLAN with that in principle?

While your switch can pass VLAN-tagged frames, it is transparrent fir VLANs, it can't manipulate VLAN tags by itself. Which is fine if all connected devices handle VLANs natively, but quite often that's not true. So in your case it's best to use it as aggregation switch for all-untagged part of network.

Secondly, someone else suggested that I should have multiple VLANs on hypervisor.
This is pretty good argument for turning your setup to VLANs.

So you'll need a few VLANs:
  1. WAN ... say it'll be VID=10 and ethernet port, connected to ISPs gear will be access port for this VLAN.
    You'll need vlan interface with vlan-ids=10 for IP stuff dealing with WAN on main router
  2. Home LAN ... say it'll be VID=20 and most ethernet portd will be access ports for this VLAN. Notable exception from this rule will be ports on hEX and hAP ac connecting each other, these two will be trunk (tagged) ports for this VLAN. You'll need vlan interface for this VLAN on main router. If you'll go with management VLAN, then you don't need vlan interface for this VID on the other MT. Member of this VLAN will be also wlan interface (at least Home LAN SSID).
  3. Management VLAN ... say it'll be VID=30. Depending on how you want to access management (from Home LAN via router with firewall or via dedicated management port on one of Mikrotiks) you'll have single access port or not. In any case, the inter-MT connection will have it tagged, the connection towards VM hypervisor as well. Ypu'll need vlan interface for this VID on both MTs as well.
  4. number of VLANs needed for different VMs (e.g. 41-45). They will be tagged on trunk towards VM hypervisor and you'll need vlan interfaces on main router ...
Every of VLANs mentioned mean separate IP subnet and main router will do traffic forwarding (or blocking) between them. Regarding firewall: default is still good starting point, but you'll have to insert a few rules to alow (or not) connectivity between different subnets.

As to your gear collection: hEX Gr3 is a very fine router but not so fine switch (when it comes to VLANs it has to process everything in CPU). I'd recommend you to use hEX as your main router and use hAP ac (you mentioned it was lite in one of your posts?) as switch and AP.

The tutorial mentioned by @anav is your next recommended reading. You can configure both devices according to tutorial. When everything works as intended, you can re-configure hAP ac to use HW offload.

As you noticed, the "old school" of VLANs came in different dialects depending on switch chip built into particular device. We'll help you with correct dialect for your hAP when the right moment comes. As I already mentioned, hEX can't do it in hardware ...

I suggest you start off with physical/VLAN setup. Suggest to start with hEX, reset it to factory defaults (to get decent firewall starting point), proceed with VLANs (don't forget to use safe mode, if something goes wrong with L2 the only way out is factory reset). For starters only do WAN and Home LAN VLANs ... so we can get firewall sorted out. You'll reconfigure hAP after you get hEX running. And later you'll add other VLANs and adjust firewall rules.
BR,
Metod
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 5129
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: How to route between a bridge and a subnet?

Sun Jun 21, 2020 3:11 pm

This calls for Baileys in my coffee. MKX recommending vlan solution ( I will refrain from the usual told you sos, and it was inevitable, and mkx now a card carrying member of the vlan collective LOL)
Vlans are just plain sexy!!
I'd rather manage rats than software. Follow my advice at your own risk! (Sob & mkx forced me to write that!)
MTUNA Certified, by the Ascerbic Llama!
 
mkx
Forum Guru
Forum Guru
Posts: 4634
Joined: Thu Mar 03, 2016 10:23 pm

Re: How to route between a bridge and a subnet?

Sun Jun 21, 2020 5:28 pm

Vlans are just plain sexy!!

Some perverts find naked antennae sexy ...

@anav, FYI, I've had VLANs in my home network before you did. I just tend to approach problems pragmatically and try to KISS ... as my late physics professor at the university used to say: don't use cannon to shoot at a sparrow. Why mess with VLANs if single interface does it. But OP mentioning VMs separated to different subnets is a game changer.
BR,
Metod
 
zespri
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 53
Joined: Sat Mar 26, 2016 1:45 pm

Re: How to route between a bridge and a subnet?

Sun Jun 21, 2020 11:11 pm

Thank you everyone, a few questions:

WAN ... say it'll be VID=10 and ethernet port, connected to ISPs gear will be access port for this VLAN.
You'll need vlan interface with vlan-ids=10 for IP stuff dealing with WAN on main router

ISP does not tag VLANs for this connection. Would not me trying to have VLAN for the ISP connection when they do not do it screw it up?

Regarding firewall: default is still good starting point, but you'll have to insert a few rules to alow (or not) connectivity between different subnets.

Would it be possible, you give an example of such rule?

I'd recommend you to use hEX as your main router and use hAP ac (you mentioned it was lite in one of your posts?) as switch and AP.

I used to have hAP ac lite, but it was not giving 1000 speed that I wanted, so it was replaced with hAP ac. I still have hAP ac lite in my drawer, but since it's slower than the rest of network I don't think it will be useful (may be just for prototyping, trying things out). I know that you explained why you recommend hEX as the main router, but my limited knowledge did not let me understand that explanation. I understand that it has something to do with HW-offload which hEX is not capable off. In which scenarios do I need said HW-offload and why? Also, since hAP ac handles my wireless network and hEX is not capable of that, this aspect will have remain the same, so it looks like what is proposed is swapping places between hEX and hAP ac. Once again, I do not understand how it helps, could you please educate me ;).

I suggest you start off with physical/VLAN setup. Suggest to start with hEX, reset it to factory defaults (to get decent firewall starting point), proceed with VLANs (don't forget to use safe mode, if something goes wrong with L2 the only way out is factory reset). For starters only do WAN and Home LAN VLANs ... so we can get firewall sorted out. You'll reconfigure hAP after you get hEX running. And later you'll add other VLANs and adjust firewall rules.

I'm keen to do that, but this will cause pretty big internet disruption, so that is probably something I have to do not earlier than next weekend. Thank you again for all the help and pointers.
 
mkx
Forum Guru
Forum Guru
Posts: 4634
Joined: Thu Mar 03, 2016 10:23 pm

Re: How to route between a bridge and a subnet?

Mon Jun 22, 2020 12:27 am

WAN ... say it'll be VID=10 and ethernet port, connected to ISPs gear will be access port for this VLAN.
You'll need vlan interface with vlan-ids=10 for IP stuff dealing with WAN on main router

ISP does not tag VLANs for this connection. Would not me trying to have VLAN for the ISP connection when they do not do it screw it up?
Your WAN would get tagged upon entering your main router and untagged on exit ... so ISP would not know anything about you using VLANs.


Regarding firewall: default is still good starting point, but you'll have to insert a few rules to alow (or not) connectivity between different subnets.

Would it be possible, you give an example of such rule?
According to example I described ... you'd have VLAN interfaces vlan20 (Home VLAN) and vlan41 (one of VMs). If you will stick with "allow needed, block the rest" approach to firewall rules, you would need a rule such as
add chain=forward in-interface=vlan20 out-interface=vlan41 action=accept
placed below "accept established,related" general rule and above the "drop all" rule at the end of filter rules. The rule would then be only evaluated for packets initiating new connection where indicated flow direction makes sense (connections initiated from Home LAN), while the rest of packets (the return packets as well) would be passed due to established/related rule.
You would need similar rule for every allowed connection (single-direction) between a pair of VLANs.

I'd recommend you to use hEX as your main router and use hAP ac (you mentioned it was lite in one of your posts?) as switch and AP.

I used to have hAP ac lite, but it was not giving 1000 speed that I wanted, so it was replaced with hAP ac.
Routing performance should be roughly the same with both hEX and hAP ac (non-lite), so you can use either of them as main router. I'd still choose hEX as main router as wireless consumes quite a few CPU cycles starving routing and firewall processes.
The original reason for me proposing to use hEX as main router: HW offload (meaning function is performed by dedicated hardware rather than general-purpose CPU offering better performance) is available for switching (passing packets between different ethernet ports, e.g. between shkaf and Fingbox), but not for routing (handling IP packets passing between different IP networks). If one device does more routing and less switching, then it is sensible to assign that role to device which is worse at switching. But it really isn't that simple, decission depends on several factors.

I'm keen to do that, but this will cause pretty big internet disruption, so that is probably something I have to do not earlier than next weekend. Thank you again for all the help and pointers.
That's why I suggested it that way ... to minimize downtime. You might actually prototype whole setup using your spare hAP ac lite ... and when things start to work (and forget about the speed for a moment), migrate the config to your "production" boxes (hAP ac). hEX is separate issue (only a switch in your case) and should be easy to configure after you're finished with way more complex configuration of main router.
BR,
Metod
 
zespri
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 53
Joined: Sat Mar 26, 2016 1:45 pm

Re: How to route between a bridge and a subnet?

Mon Jun 22, 2020 10:53 am

Thank you!

WAN ... say it'll be VID=10 and ethernet port, connected to ISPs gear will be access port for this VLAN.
You'll need vlan interface with vlan-ids=10 for IP stuff dealing with WAN on main router

ISP does not tag VLANs for this connection. Would not me trying to have VLAN for the ISP connection when they do not do it screw it up?
Your WAN would get tagged upon entering your main router and untagged on exit ... so ISP would not know anything about you using VLANs.

I've been reading this VLAN link that was posted above and none of the examples have VLAN for WAN. Only for MGMT/BASE, Home, Guest, etc. In all the examples on that thread WAN is on a separate interface that is not part of a bridge and it is not VLAN configured. What makes my case different?
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 5129
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: How to route between a bridge and a subnet?

Mon Jun 22, 2020 4:02 pm

Not sure what you mean, but for example I have many vlans at my location. MY ISP internet also comes in on a VLAN.

That ISP vlan is identified in my interface vlan settings and the interface for the vlan is the etherport my ISP is coming in on. thats IT!
Oh I make sure to include my vlan on the in-interface-list for WAN.
I'd rather manage rats than software. Follow my advice at your own risk! (Sob & mkx forced me to write that!)
MTUNA Certified, by the Ascerbic Llama!
 
zespri
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 53
Joined: Sat Mar 26, 2016 1:45 pm

Re: How to route between a bridge and a subnet?

Mon Jun 22, 2020 10:42 pm

Not sure what you mean, but for example I have many vlans at my location. MY ISP internet also comes in on a VLAN.

That ISP vlan is identified in my interface vlan settings and the interface for the vlan is the etherport my ISP is coming in on. thats IT!
Oh I make sure to include my vlan on the in-interface-list for WAN.
If you not sure what I mean, please let me know what is unclear. I'm asking about specific statement mkx made, which I'm quoting above. What you are saying makes sense but it does not answer the question.
 
mkx
Forum Guru
Forum Guru
Posts: 4634
Joined: Thu Mar 03, 2016 10:23 pm

Re: How to route between a bridge and a subnet?

Tue Jun 23, 2020 12:03 am

It is like this: router forwards IP packets between different sub-networks and WAN is one of those sub-networks, there's nothing really magical about it. Ideally those sub-networks don't share L2 (e.g. ethernet) infrastructure. However VLANs offer additional layer (sometimes called L2.5) of separation and using that different IP sub-networks can share same L2 infrastructure. The beauty of it is that VLANs have borders ... by means of access ports. Which means that you can nicely use VLAN as underlying infrastructure for WAN sub-network internally to your infrastructure which ends with access port, connected to ISP's modem.
Another view on VLAN: the acronym stands for virtual LAN, where local originally stood for ethernet (or arcnet or whatever technology for local network), but nowdays it should stand for virtual network. In practice VLANs are used instead of physical wires and same UTP patch cable can carry either WAN or LAN packets, depending which devices it connects.

I can understand you're uncomfortable with new and unknown concepts. The beauty is that you can test the concept with the spare device you have. Try it, test it ... in worst case nothing will work, in best case it'll work just fine.
BR,
Metod
 
zespri
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 53
Joined: Sat Mar 26, 2016 1:45 pm

Re: How to route between a bridge and a subnet?

Tue Jun 23, 2020 12:23 am

I can understand you're uncomfortable with new and unknown concepts. The beauty is that you can test the concept with the spare device you have. Try it, test it ... in worst case nothing will work, in best case it'll work just fine.

Thanks, I think I'm getting the hang of it. It's just taking time.

It is like this: router forwards IP packets between different sub-networks and WAN is one of those sub-networks, there's nothing really magical about it. Ideally those sub-networks don't share L2 (e.g. ethernet) infrastructure. However VLANs offer additional layer (sometimes called L2.5) of separation and using that different IP sub-networks can share same L2 infrastructure. The beauty of it is that VLANs have borders ... by means of access ports. Which means that you can nicely use VLAN as underlying infrastructure for WAN sub-network internally to your infrastructure which ends with access port, connected to ISP's modem.

I think what you are saying, is that the port we are connecting to ISP, the yellow WAN port on the diagrams for that link, can be an access port. The packets get tagged on this port during ingress and untagged during egress (providing that ISP LAN itself is not tagged). This way you will have a separate "WAN" VLAN.

All the examples at that link do NOT have a separate WAN VLAN, and it does not look like the WAN interface is an access port in these examples.

You are saying, that by making it into an access port and creating a VLAN for that we are getting better separation. Did I understand this correctly?

Thank you for you patience!
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 5129
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: How to route between a bridge and a subnet?

Tue Jun 23, 2020 4:22 am

Why would you want to setup a vlan to your WAN port or ISP, if they are not sending data down a vlan to you??
If they are sending data on a vlan, there is nothing you have to do regarding bridge ports or bridge vlan filtering, no nonsense of ingress or egress
You simply identify to the router that the vlan is associates with etherX when you define the VLAN.

?? I have no clue why Mkx is cluttering up our brains with theoretical mushiness??
I'd rather manage rats than software. Follow my advice at your own risk! (Sob & mkx forced me to write that!)
MTUNA Certified, by the Ascerbic Llama!
 
zespri
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 53
Joined: Sat Mar 26, 2016 1:45 pm

Re: How to route between a bridge and a subnet?

Tue Jun 23, 2020 4:39 am

Why would you want to setup a vlan to your WAN port or ISP, if they are not sending data down a vlan to you??
Yep, that's pretty much what I'm trying to understand.
 
zespri
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 53
Joined: Sat Mar 26, 2016 1:45 pm

Re: How to route between a bridge and a subnet?

Tue Jun 23, 2020 8:52 am

Okay, next iteration of thinking.


Image

This is my idea about vlans ports and bridges. Please let me know if this makes sense

Image

Following the idea of configuring hEX as the dumb switch and having trunk port configured on the Dell Precision hypervisor.
  • Home - This is where most of the stuff on home network will live. wlan1 and wlan2 is 2/5GHz.
  • Guest - this is for guest Wifi
  • Esxi - Management interface for Esxi
  • VMInt - VMs for local network
  • VMExt - VMs to be exposed externally
Now note that I put down 2 bridges here. The reason is I think that from VPN user perspective bridge is all you can specify to restrict a user.
I'm not entirely sure how the routing between the two bridges will work and if it is possible in principle.

I would like to know:
Does this make sense? How do I configure routing between bridges?
Since I'm not very familiar yet with how hypervisor does trunking, I think I might use the third mikrotik for testing this out. I will configure trunk port on it similar to BR-ESXI on the main one, and plug a raspberry pi (or laptop) to another port on that router and see if I get the connection. If it works this way, I an start messing with esxi.
 
mkx
Forum Guru
Forum Guru
Posts: 4634
Joined: Thu Mar 03, 2016 10:23 pm

Re: How to route between a bridge and a subnet?

Tue Jun 23, 2020 11:19 am

I think what you are saying, is that the port we are connecting to ISP, the yellow WAN port on the diagrams for that link, can be an access port. The packets get tagged on this port during ingress and untagged during egress (providing that ISP LAN itself is not tagged). This way you will have a separate "WAN" VLAN.
Nice thing about Mikrotik devices is that you can completely reconfigure them ... meaning that there are no ports with dedicated role per-se. For example, you can reconfigure your unit to use ether3 as WAN port. This can be confusing because it no longer corresponds to markings on the device's case, but as long as you understand the configuration this doesn't really matter.

When moving config towards VLAN, one can do it all the way so that internally to VLAN-capable gear (in your case it's both Mikrotik devices) everything will be VLAN-tagged ... including WAN. At the first sight this doesn't make much sense, but actually simplifies configuration ... in a way where all IP stuff is configred on top of VLAN interfaces. Additionally this brings possibility to run router/firewall on device which is not physically connected directly to the ISP device.
E.g. in one of my SOHO installations, my xDSL modem is connected to managed switch and I'm using RB as router-on-a-stick (which means it only has single ethernet cable connection to the main switch), all sub-nets (and WAN is only one of "sub-nets") are handled via VLANs. Further: if ISP is providing more than just internet (e.g. IPTV, VoIP), those additional services might come in tagged VLANs (while internet might come as untagged). In this case making WAN port a hybrid (or trunk) and tag internet packets on ingress (and untag them on egress) via mechanism of access port actually allows one to flexibly route/switch the supplementary services as well.

But as @anav noted: if you don't want to fuss with WAN interface, simply connect it to ether1 of main router, then there's no need in converting WAN to VLAN internally.
BR,
Metod
 
zespri
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 53
Joined: Sat Mar 26, 2016 1:45 pm

Re: How to route between a bridge and a subnet?

Tue Jun 23, 2020 12:02 pm

Nice thing about Mikrotik devices is that you can completely reconfigure them ... meaning that there are no ports with dedicated role per-se. For example, you can reconfigure your unit to use ether3 as WAN port. This can be confusing because it no longer corresponds to markings on the device's case, but as long as you understand the configuration this doesn't really matter.
Thank you. I don't think that this confuses me. when I referred to the yellow port, I did not mean marking on the device's case, I referred to a diagram at the link above that was your suggested reading. The rest of your post seems also clear to me.

At the moment I'm trying to figure out what VLANs I'm going to have (as per above, some questions there) and I'm running example configurations from the VLANs link to familiarise myself with how it is all working. The guide looks quite clear and my experiments match expectations so far. The challenging part will be what's not covered in these examples.
 
mkx
Forum Guru
Forum Guru
Posts: 4634
Joined: Thu Mar 03, 2016 10:23 pm

Re: How to route between a bridge and a subnet?

Tue Jun 23, 2020 12:25 pm

when I referred to the yellow port, I did not mean marking on the device's case, I referred to a diagram at the link above that was your suggested reading.

Right ... and I'm trying to explain that you can convert yellow WAN port to access port of yet another VLAN (let's randomly pick yellow colour for this VLAN) and then configure WAN-specific IP stuff on top of yellow VLAN. But, as I said, you don't have to ... probably easier to ingest all the stuff if you stick to tutorial as closely as it gets.

Nice thing about VLANs is that you can actually start off with two (just to have more than one) and after you're familiar with VLAN configuration, simply add other VLANs you need and (almost) copy-paste needed config. I.e. your VM VLANs will be almost identical, they will share physical infrastructure (trunk port towards hypervisor and main router), they will differ only with regard to VLAN ID and IP addressing. Firewall filter rules will probably be very similar again.

I think that the hard part for you is to get VLANs done right ... the access and trunk ports. After that it'll be easy to configure IP (firewall, ...) because you won't need to think about physical stuff any more, it'll be some clouds (each representing one sub-net). If you get into trouble at any time, just come back, post full configuration at the moment, and we'll help you get further.
BR,
Metod

Who is online

Users browsing this forum: ruwerra, wichets and 47 guests