VLAN between two routers. Can it work!? If so how?

RouterOS Beginner Basics
1

Hello,

The title already explain my problem. My main router (pfSense) is connected with my new Mikrotik CRS317, via VLAN’s. I Always did expect that would not be any problem, but it is.

Situation is as follows:

  • there are two VLAN’s between pfSense and the CRS317. One for management. One for data.
  • pfSense should be “the owner” of the VLAN’s and is gateway for those VLAN’s,
  • the CRS317 should only forward the VLAN’s towards the indicated ports and
  • in case of the MNGT-lan the CPU should be reachable via that management VLAN.
  • mngt vlans is the default gateway for mngt, data vlan is the default gateway for data

That easy …… I thought.

However, the CRS317 assumes that it own’s every VLAN and forces a (extra) gateway on that VLAN, of course that does not work!

What I want from the CRS is that:

  • it is gateway (and DHCP-server) for local only VLAN’s and
  • that it just client of incoming VLAN’s (learing GW, address and DNS from the broadcasts)

I tried a lot to get that to work:

  • Did add address ranges to IP addresses with as address “2” where “1” is the external gateway on pfSense

  • Adding the address range to IP routes

  • Tried using IP address “1” local and “remote” which is of course conflicting :frowning:

  • Adding default routes to the routing table (using separate routing table mark for management)

  • Or, and that is IMHO the correct option(!), not at all defining addresses and routes, since that is responsibility of pfSense.
    …. Only adding the default routes to the routing table.

Whatever none of these methods work. So please advice on this.

Sincerely,

Louis

2

You wrote long description, but still nobody can have any idea (except guesses), what you actually did on CRS. I’ll borrow a signature from one other user, as it fits perfectly here:

Instead of writing novels, post /export hide-sensitive. Use find&replace in your favourite text editor to systematically replace all occurrences of each public IP address potentially identifying you by a distinctive pattern such as my.public.ip.1.

3

Did you read the wiki about the bridge chip implementation?

4

Hello,

Perhaps the picture at the botum helps

pfSense and the existing network to handle 1G devices. CRS317 as high speed core.

Intention:

  • two vlans to connect them (data and mngt)
  • the gateway is at the pfSense side
  • the local high speed vlans’s purple, have the gate way on the CRS317 side
  • the management lan should have access to the cpu/gui
  • the datalan is also gateway for the purple high speed lans

My problem is that the behavior of both routers interfere.

Hope this helps to understand the situation.

I tried a lot of configs, at this moment

  • the datalan:
  • defined as default route under ip routers
  • not defined under ip addresses
  • dhcp client referring to pfsense gw
  • bridge port is not included in the vlan
  • perhaps there need to be a gateway at the CRS-side to interconnect the FW
  • the management lan
  • mngt lan additional has an ip-address ranges assigned in order to access the CPU
  • that creates a gateway at two sides, but it works
    (with e.g. as problem that data can leave the vlan !! very unpleasant !)

So I do not have a good solution, I hope that someone has!

Sincerely.

Louis
NetworkTwoRoutersHighLevel.PNG

5

Hello,

Short reaction on the question “Did you read the wiki about the bridge chip implementation?”. Yep I did. And on your advice today I did read again.
(I assume you refered to https://wiki.mikrotik.com/wiki/Manual:Switch_Chip_Features)

In fact I did read a lot !! Far more than should be necessay IMHO. RouterOS is very powerfull, however the GUI …. hum. For example, IMHO there should be a separate item to assign addresses to " , VLAN, IPV4 -, IPV6-address. Address asignment not hided under IP address, IP V6 etc. I also do not like that there is no differentiation between Bride-port and CPU-port. Etc.

Louis
6

Hey

CRS is not a router, so you shouldn’t be using it as one.

I would suggest to upgrade the pfsense to “the only router” status:

  • only bridge on CRS for “data” vlans → you did say that pfsens is owner of these! if so, CRS should not route (nor firewall)
  • this means no ip on data vlans for CRS
  • only ip it should have is on the mgmt vlan, with default route there

this will solve all your problems :wink:

7

Tja,

The described configuration had a couple of reasons:

  • to keep the high speed datastreams away form pfSense (intel Pentium)
  • to see if it was an option to use the internal router in state of pfSense
  • no pfSense computer upgrade
  • no pc saves place and energy
  • to save on 10G card for pfSense
  • to save interfaces between pfSense and the CRS317
  • to have a look at the router and firewall performance of the CRS
  • so, yesterday I did a very basic speedtest to the only two 10G devices I have at the moment, the NAS and my PC.
  • Both connected to the CRS317. Both on another VLAN
  • FW having a very, very minimal set or rules
  • So I did start a simple filetransfer …….. and was quite, quite disapointed.
  • the transfer speed was about 250 mbits/s ……. for a router switch with port-speed of 10G (thats 1/40 of one single port)
  • I know that there are fast path options, I tried it, but my actual knowledge agbout that is minimal
  • ….. an very small improvement , perhaps I was doing something wrong (this is not even the capacity of a cheap home router)

  • As private user, I do not need extreme FW/routing capactity, however lets say, something like 5 Gbit (20 times more!, would be nice)

  • I can not understand Mikrotik is selling that kind of routing capacity with this kind of switch (a shame!!)

So, it is irritating me(!) that I cannot make the described config working in a proper way, but at the end I agree completely!

So, it is irritating me that I cannot make the described config working, but at the end I agree completely!
However, serious disadvantages:

  • extra pc (pf-sense)
  • a few hundred € for the ports, network card and perhaps new CPU

Sincerely,

Louis

8

If the PCC and NAS are same vlan, then your config is wrong

9
  • to keep the high speed datastreams away form pfSense (intel Pentium)
  • to see if it was an option to use the internal router in state of pfSense
    → a CRS can’t route 10g of data either!
    → not with a CRS

    \

  • to save interfaces between pfSense and the CRS317
    → don’t understand that one

  • to have a look at the router and firewall performance of the CRS
    → it’s bad, see the cpu specs…

An advice maybe: keep all your 10g devices in same ip range, so you don’t have to route that traffic and can bridge at wire-speed

If you chose to go for upgrade, it shouldn’t be that expensive: an i3-9xxx + single 10g interface (with vlans ontop) will do

10

So you took your brand new Ferrari and went on to plow the field.

If NAS and PC are in different VLANs, then transfers between tgem involve routing. If you checked the official test results, looking in “routing” row under “ethernet test results”, you’d see that in lab environment (and copying files using SMB protocol is not) routing throughput is somewhere around 500Mbps.

In short: CRS devices are decent (good even) switches but suck big time as routers.