Cannot get simple VLAN to work on CRS317 ☹

Louis2 · Sat Aug 31, 2019 10:50 pm

Hello,

I have a pfSense router connected to one of the ports of a CRS317 (running RouterOS) . Within that connection a VLAN-A. On one of the CRS317 ports having VLAN-ID-A a PC.

Pf Sense is the master of that VLAN
- Has the IPV4 gateway 192.168.x.1 and the IPV6 gateway
- A certain IPV4 range 192.168.x/24 and a certain IPV6 range 2001:axcd:axcd: x ::1 /64 assigned
- IPV4 DHCP server running, IPV6 DHCP and RA server running

On the CRS317
- VLAN-A with
o In comming pfsense-trunk (tagged)
o Some other port connected with a PC (untagged), having pivd “A”
- Interfaces vlan: pfsense-trunk, vlan-id “A”, name “testvlan”

Bridge
- Vlan filtering enable (of course)
- port:
o Pfsense-trunk belongs to bridge
o PC-port belongs to bridge
- vlan:
o Testvlan: vlan-id-A, pfsense-trunk (tagged), pcport (untagged)

Not so complicated IMHO, should work without any problems (!?) ….. however it is not working!

- The PC get IPV4 and IPV6 addresses assigned from pfsense DHCP-servers (perfect)
- The PC can not ping the pfSense gateway using IPV4 (I am lost !!)
- The PC can ping the pfSense gateway using IPV6 (perfect)

Two questions:
1) I have to admit, that I do not at all understand why there is a VLAN section under menu-item "interface". Can some one explain, what is the goal?
(There is a vlan section under bridge, what IMHO should do !!??)
2) Can someone explain why the IPV4-ping is not working, where the IPV6-ping and DHCP are both working!!??
Sincerely,

Louis

mkx · Sun Sep 01, 2019 12:08 am

Two questions:
1) I have to admit, that I do not at all understand why there is a VLAN section under menu-item "interface". Can some one explain, what is the goal?
(There is a vlan section under bridge, what IMHO should do !!??)
2) Can someone explain why the IPV4-ping is not working, where the IPV6-ping and DHCP are both working!!??

/interface vlan allows device to interact with a VLAN. If, for example, you wanted to give CRS IP address in VLAN A (e.g. for management purposes), you would create an interface in this configuration section ... then you'd use the newly created interface to configure IP address and possibly bind some services (e.g. DHCP server)
As I wrote in your other thread: post outout of /export hide-sensitive so we can see your actual configuration. Without seeing it, we can just guess ...

Louis2 · Sun Sep 01, 2019 8:57 pm

Ok,

Thanks for your answer on my first question "Why vlan option in interface".

I was thinking in that direction, however it does hardly fit in the way I am thinking:
- step 1 you have a device (in this case an interface)
- step 2 you assign an IPV4 and IPV6 IP on ethernet level
- step 3 you assign an vlan to that interface, that stil fit's in my way of thinking, however why "one" whay not "20" !!!??

and then, the next step is completely missing
- step 4 you assign per vlan an IPV4 and IPV6 IP and network mask

It is probably all possible, but in a very wired way (IMHO)

But I will look at the interfaces, with what you are saying in mind!

Sincerely,

Louis

Louis

mkx · Sun Sep 01, 2019 9:54 pm

When one adds VLANs to ethernet, another layer gets added ... which is above ethernet and below IP. To reflect that, in ROS one creates VLAN interfaces on top of physical interface(s) and sets IP stuff on those VLAN interfaces.

Conceptual complication is if one uses hybrid ports, i.e. both tagged and untagged frames over same port. In this case IP stuff can be set directly on ether port but only for the untagged frames. Untagged can, in this case, be considered as special VLAN, but is exception in how things are configured (hence my claim about it being a complication).

Louis2 · Sun Sep 01, 2019 10:43 pm

Sorry,

I made a mistake in my replay. As expected you can assign multiple vlan's to one interface.

And in your reaction, I see a very interesting remark, in the second sentence.

"Conceptual complication is if one uses hybrid ports, i.e. both tagged and untagged frames over same port. In this case IP stuff can be set directly on ether port but only for the untagged frames. Untagged can, in this case, be considered as special VLAN, but is exception in how things are configured (hence my claim about it being a complication)."

You are right that the PIVD is directly on the ethernet. But as I have Always looked at that is like a kind of "front portal / door".
- as soon as an untagged frame enters the ethernet port, it gets the PIVD and from that moment on, it is just like a tagged frame member of that particular VLAN
- and it stays part of that VLAN as long as it is in the network.
- and at the end the tag can eventually be removed, for devices not supporting vlan's (windows PC's

)

Not 100% sure that is the way the CRS317 is processing things!

I have reed things, which make me think that:
- the crs dynamically creates a vlan if there is a PIVD related to a non-existing VLAN-id (IMHO it should throw the package away)
- and it is auto connecting all ports having the same PIVD when there is no vlan defined at all (also disgusting IMHO)
But … perhaps conform specs ….

Louis

mkx · Sun Sep 01, 2019 11:20 pm

There are two things which are connected, but in ROS configured more or less independently:

membership of ports on VLANs ... either tagged or untagged with PVID set.
port security
For each port it is possible to set options which define which kind of frames are allowed on ingress. Options are ingress-filtering and frame-types. Default setting is permissive though.

It might be true that ROS somehow implicitly deals with PVID-defined VLANs. However, I don't think that's the case, specially if one strenghtens VLAN security by setting ingress-filtering=yes and setting frame-types= to appropriate value. The former setting means that only frames with VID matching one of egress VIDs are allowed on ingress. Hence no implicit VLANs ...

Louis2 · Tue Sep 03, 2019 9:29 pm

Hello,

I am making some progress …….. that is to say ….. I know what is going wrong ……

I discovered IMHO a very obscure CRS behavoir.
- I can ARP ping the other side (the gateway), using the ping tool (arp ping) and the test pc
- but I can not (IP) ping the other side !!

So level-2 is OK, but I can hardly believe it, level-3 is blocked!

And that on what is supposed a extremely simple and transparant VLAN!
Never seen something that bizar before!

Note that the test vlan is comming from else where (pfSense). pfSense, is providing GW and DHCP server
CRS is just having one tagged interface, connected with pfSense and one untagged interface connected with a PC
- bridge interface not included (should be transaparant)
- no IP addresses
- no routes
- just a pivd at the test PC-port

Running latest very RouterOS

Ok now that I know, what the problem is, it will probably be easier to find the solution !!

Sincerely.

Louis

CZFan · Tue Sep 03, 2019 9:42 pm

Hello,

I am making some progress …….. that is to say ….. I know what is going wrong ……

I discovered IMHO a very obscure CRS behavoir.
- I can ARP ping the other side (the gateway), using the ping tool (arp ping) and the test pc
- but I can not (IP) ping the other side !!

So level-2 is OK, but I can hardly believe it, level-3 is blocked!

And that on what is supposed a extremely simple and transparant VLAN!
Never seen something that bizar before!

Note that the test vlan is comming from else where (pfSense). pfSense, is providing GW and DHCP server
CRS is just having one tagged interface, connected with pfSense and one untagged interface connected with a PC
- bridge interface not included (should be transaparant)
- no IP addresses
- no routes
- just a pivd at the test PC-port

Running latest very RouterOS

Ok now that I know, what the problem is, it will probably be easier to find the solution !!

Sincerely.

Louis

Sorry, I am struggling understand this, if no IP config on the device, how is it suppose to "ping" on IP layer?

Louis2 · Tue Sep 03, 2019 10:30 pm

That,

is quite simple!
- the CRS is intended as just a managed L2 switch supporting VLAN's, just L2-transport layer
- like every simple managed L2/L3 switch you can buy
- each device has a either a static IP-address or it gets one from the DHCP-server (situated in pfSense)
- that works via ARP-broadcasts and from that each device knows the MAC of the Gateway and its IP-address
- using that MAC the L2-switch can route the L2-package to its (IP and MAC) destination, without knowing any thing about IP
- the VLAN is there to create a virtual L2-channel, within the "physical ethernet-pipe"

Sincerely.

Louis

mkx · Tue Sep 03, 2019 11:18 pm

I strongly suggest you to post current config on CRS, export it by executing command /export (do include leading '/'). Without that we can play whack-a-mole for days without any result, because I can not imagine how you implemented in ROS what you described in your previous posts...

Louis2 · Wed Sep 04, 2019 9:52 am

Below my actual testconfig,

At this moment I am concentrating on getting the simple test L2-VLAN123 to work. To be sure that there is no whatever other problem, I disabled the link to my gs1920 and removed a couple om mangle rules. VLAN123 comming from pdSense is entering the switch via interface "04 RouterData" (tagged) and the test-PC is at interface "01 GLASS-SW" (untagged pivd 123).

But even in this extreme simple set-up L2-arp is working, but IP-ping is blocked.

Sincererly,

Louis

# sep/04/2019 08:36:02 by RouterOS 6.45.5
# software id = UT7L-U4J9
#
# model = CRS317-1G-16S+
# serial number = xyz
/interface bridge
add admin-mac=xyz auto-mac=no comment=defconf \
    ingress-filtering=yes name=VirtualSwitch1 vlan-filtering=yes
/interface ethernet
set [ find default-name=sfp-sfpplus1 ] advertise=100M-full,1000M-full name=\
    "01 GLASS-SW"
set [ find default-name=sfp-sfpplus2 ] advertise=100M-full,1000M-full \
    disabled=yes name="02 TBD"
set [ find default-name=sfp-sfpplus3 ] advertise=100M-full,1000M-full \
    disabled=yes name="03 RouterMngt"
set [ find default-name=sfp-sfpplus4 ] advertise=100M-full,1000M-full name=\
    "04 RouterData"
set [ find default-name=sfp-sfpplus5 ] advertise=100M-full,1000M-full \
    disabled=yes name="05 GS1920"
set [ find default-name=sfp-sfpplus6 ] advertise=100M-full,1000M-full \
    disabled=yes name="06 FB-L2_PC_L"
set [ find default-name=sfp-sfpplus7 ] advertise=100M-full,1000M-full \
    disabled=yes name="07 SW-woonkamer"
set [ find default-name=sfp-sfpplus8 ] advertise=100M-full,1000M-full \
    disabled=yes name="08 SW-logeerkamer"
set [ find default-name=sfp-sfpplus9 ] advertise=100M-full,1000M-full \
    disabled=yes name="09 SW-SLK-L&N\""
set [ find default-name=sfp-sfpplus10 ] advertise=100M-full,1000M-full \
    disabled=yes name="10 SW-werkkamer"
set [ find default-name=sfp-sfpplus11 ] advertise=100M-full,1000M-full name=\
    "11 NAS_EM0"
set [ find default-name=sfp-sfpplus12 ] advertise=1000M-full,10000M-full \
    name="12 NAS_DATA"
set [ find default-name=sfp-sfpplus13 ] advertise=\
    100M-full,1000M-full,2500M-full,5000M-full,10000M-full name="13 Server"
set [ find default-name=sfp-sfpplus14 ] advertise=100M-full,1000M-full \
    disabled=yes name="14 KVM-Link"
set [ find default-name=sfp-sfpplus15 ] advertise=100M-full,1000M-full \
    disabled=yes name="15 S-Elise_LA"
set [ find default-name=sfp-sfpplus16 ] advertise=\
    100M-full,1000M-full,2500M-full,5000M-full,10000M-full name=\
    "16 PC-werkkamer"
set [ find default-name=ether1 ] advertise=100M-full,1000M-full name=\
    "17 LOC-MNGT"
/interface vlan
add interface=VirtualSwitch1 name=DEFAULT-LAN vlan-id=1
add interface="05 GS1920" name=GreenZone vlan-id=18
add interface=VirtualSwitch1 name="GreenZone 10G" vlan-id=218
add interface="05 GS1920" name=GuestLAN vlan-id=26
add interface="05 GS1920" name=IOT-LAN vlan-id=13
add interface="05 GS1920" name=IPTV vlan-id=4
add interface="05 GS1920" name=Internet vlan-id=6
add interface="05 GS1920" name=KVM vlan-id=50
add interface=VirtualSwitch1 name=MNGT-LAN vlan-id=10
add interface="05 GS1920" name=PC-LAN vlan-id=16
add interface=VirtualSwitch1 name="PC-LAN 10G" vlan-id=216
add interface="05 GS1920" name=RedZone vlan-id=14
add interface="04 RouterData" name=Route99 vlan-id=99
add interface=VirtualSwitch1 name=VLAN88 vlan-id=88
add interface="04 RouterData" name=VLAN123 vlan-id=123
add interface="05 GS1920" name=VoIP vlan-id=7
/interface list
add name=WAN
add name=LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip hotspot profile
set [ find default=yes ] html-directory=flash/hotspot
/ip pool
add name=PCLAN_POOL ranges=192.168.216.128-192.168.216.253
add name=GZ_POOL ranges=192.168.218.128-192.168.218.253
add name=VL88_POOL ranges=192.168.88.128-192.168.88.253
/ip dhcp-server
add address-pool=PCLAN_POOL disabled=no interface="PC-LAN 10G" name=\
    PCLAN_DHCP
add address-pool=GZ_POOL disabled=no interface="GreenZone 10G" name=GZ_DHCP
add address-pool=VL88_POOL disabled=no interface=VLAN88 name=VL88_DHCP
/routing bgp instance
set default as=65456 out-filter=connected-in router-id=192.168.99.201
/interface bridge port
add bridge=VirtualSwitch1 comment=defconf ingress-filtering=yes interface=\
    "17 LOC-MNGT" pvid=88
add bridge=VirtualSwitch1 comment=defconf edge=no interface="01 GLASS-SW" \
    pvid=123
add bridge=VirtualSwitch1 comment=defconf edge=no frame-types=\
    admit-only-vlan-tagged ingress-filtering=yes interface="02 TBD" pvid=1002
add bridge=VirtualSwitch1 comment=defconf edge=no frame-types=\
    admit-only-vlan-tagged ingress-filtering=yes interface="03 RouterMngt" \
    pvid=1003
add bridge=VirtualSwitch1 comment=defconf edge=no frame-types=\
    admit-only-vlan-tagged ingress-filtering=yes interface="04 RouterData" \
    pvid=1004
add bridge=VirtualSwitch1 comment=defconf edge=no frame-types=\
    admit-only-vlan-tagged ingress-filtering=yes interface="05 GS1920" pvid=\
    1005
add bridge=VirtualSwitch1 comment=defconf edge=yes frame-types=\
    admit-only-untagged-and-priority-tagged ingress-filtering=yes interface=\
    "06 FB-L2_PC_L" pvid=16
add bridge=VirtualSwitch1 comment=defconf edge=no frame-types=\
    admit-only-vlan-tagged ingress-filtering=yes interface="07 SW-woonkamer" \
    pvid=1007
add bridge=VirtualSwitch1 comment=defconf edge=no frame-types=\
    admit-only-vlan-tagged ingress-filtering=yes interface=\
    "08 SW-logeerkamer" pvid=1008
add bridge=VirtualSwitch1 comment=defconf edge=no frame-types=\
    admit-only-vlan-tagged ingress-filtering=yes interface="09 SW-SLK-L&N\"" \
    pvid=1009
add bridge=VirtualSwitch1 comment=defconf edge=no frame-types=\
    admit-only-vlan-tagged ingress-filtering=yes interface="10 SW-werkkamer" \
    pvid=1010
add bridge=VirtualSwitch1 comment=defconf edge=no frame-types=\
    admit-only-vlan-tagged ingress-filtering=yes interface="11 NAS_EM0" pvid=\
    1011
add bridge=VirtualSwitch1 comment=defconf edge=no frame-types=\
    admit-only-vlan-tagged ingress-filtering=yes interface="12 NAS_DATA" \
    pvid=1012
add bridge=VirtualSwitch1 comment=defconf edge=yes frame-types=\
    admit-only-untagged-and-priority-tagged ingress-filtering=yes interface=\
    "13 Server" pvid=16
add bridge=VirtualSwitch1 comment=defconf edge=yes frame-types=\
    admit-only-untagged-and-priority-tagged ingress-filtering=yes interface=\
    "14 KVM-Link" pvid=50
add bridge=VirtualSwitch1 comment=defconf edge=yes frame-types=\
    admit-only-untagged-and-priority-tagged ingress-filtering=yes interface=\
    "15 S-Elise_LA" pvid=16
add bridge=VirtualSwitch1 comment=defconf edge=yes frame-types=\
    admit-only-untagged-and-priority-tagged ingress-filtering=yes interface=\
    "16 PC-werkkamer" pvid=216
/ip neighbor discovery-settings
set discover-interface-list=all
/interface bridge vlan
add bridge=VirtualSwitch1 comment=MNGT-LAN tagged=\
    "05 GS1920,11 NAS_EM0,VirtualSwitch1,03 RouterMngt" vlan-ids=10
add bridge=VirtualSwitch1 comment="PC-LAN 10G" tagged=\
    "PC-LAN 10G,VirtualSwitch1" untagged="16 PC-werkkamer" vlan-ids=216
add bridge=VirtualSwitch1 comment="GreenZone 10G" tagged=\
    "12 NAS_DATA,VirtualSwitch1" vlan-ids=218
add bridge=VirtualSwitch1 comment=IPTV tagged="05 GS1920" vlan-ids=4
add bridge=VirtualSwitch1 comment=Internet tagged="05 GS1920" vlan-ids=6
add bridge=VirtualSwitch1 comment="DEFAULT LAN" tagged="VirtualSwitch1,05 GS19\
    20,07 SW-woonkamer,08 SW-logeerkamer,09 SW-SLK-L&N\",10 SW-werkkamer" \
    vlan-ids=1
add bridge=VirtualSwitch1 comment=VoIP tagged="05 GS1920" vlan-ids=7
add bridge=VirtualSwitch1 comment=IOT-LAN tagged="05 GS1920" vlan-ids=13
add bridge=VirtualSwitch1 comment=RedZone tagged="05 GS1920,13 Server" \
    vlan-ids=14
add bridge=VirtualSwitch1 comment=GreenZone tagged="05 GS1920,12 NAS_DATA" \
    vlan-ids=18
add bridge=VirtualSwitch1 comment=KVM-Link tagged="05 GS1920" untagged=\
    "14 KVM-Link" vlan-ids=50
add bridge=VirtualSwitch1 comment=PC-LAN disabled=yes tagged="05 GS1920" \
    untagged="16 PC-werkkamer,06 FB-L2_PC_L,15 S-Elise_LA" vlan-ids=16
add bridge=VirtualSwitch1 comment="Route99 CRS317 <> pfSense DataGW" tagged=\
    "04 RouterData" vlan-ids=99
add bridge=VirtualSwitch1 comment="VLAN88 Local MNGT" tagged=VirtualSwitch1 \
    untagged="17 LOC-MNGT" vlan-ids=88
add bridge=VirtualSwitch1 comment=VLAN123 tagged="04 RouterData" untagged=\
    "01 GLASS-SW" vlan-ids=123
/interface list member
add interface="17 LOC-MNGT" list=LAN
add interface="01 GLASS-SW" list=LAN
add interface="02 TBD" list=WAN
add interface="03 RouterMngt" list=LAN
add interface="04 RouterData" list=LAN
add interface="05 GS1920" list=LAN
add interface="06 FB-L2_PC_L" list=LAN
add interface="07 SW-woonkamer" list=LAN
add interface="08 SW-logeerkamer" list=LAN
add interface="09 SW-SLK-L&N\"" list=LAN
add interface="10 SW-werkkamer" list=LAN
add interface="11 NAS_EM0" list=LAN
add interface="12 NAS_DATA" list=LAN
add interface="13 Server" list=LAN
add interface="14 KVM-Link" list=LAN
add interface="15 S-Elise_LA" list=LAN
add interface="16 PC-werkkamer" list=LAN
/ip address
add address=192.168.218.1/24 interface="GreenZone 10G" network=192.168.218.0
add address=192.168.216.1/24 interface="PC-LAN 10G" network=192.168.216.0
add address=192.168.10.9/24 interface=MNGT-LAN network=192.168.10.0
add address=192.168.88.1/24 interface=VLAN88 network=192.168.88.0
/ip dhcp-client
add dhcp-options=hostname,clientid disabled=no interface=MNGT-LAN
add dhcp-options=hostname,clientid disabled=no interface=Route99
/ip dhcp-server lease
add address=192.168.216.35 mac-address=06:B3:11:3C:4F:E0 server=PCLAN_DHCP
/ip dhcp-server network
add address=192.168.88.0/24 dns-server=192.168.88.1 gateway=192.168.88.1 \
    netmask=24
add address=192.168.216.0/24 dns-server=192.168.216.1 gateway=192.168.216.1
add address=192.168.218.0/24 dns-server=192.168.218.1 gateway=192.168.218.1
/ip firewall address-list
add address=0.0.0.0/8 comment="Self-Identification [RFC 3330]" list=bogons
add address=10.0.0.0/8 comment="Private[RFC 1918] - CLASS A # Check if you nee\
    d this subnet before enable it" disabled=yes list=bogons
add address=127.0.0.0/8 comment="Loopback [RFC 3330]" list=bogons
add address=169.254.0.0/16 comment="Link Local [RFC 3330]" list=bogons
add address=172.16.0.0/12 comment="Private[RFC 1918] - CLASS B # Check if you \
    need this subnet before enable it" list=bogons
add address=192.168.0.0/16 comment="Private[RFC 1918] - CLASS C # Check if you\
    \_need this subnet before enable it" disabled=yes list=bogons
add address=192.0.2.0/24 comment="Reserved - IANA - TestNet1" list=bogons
add address=192.88.99.0/24 comment="6to4 Relay Anycast [RFC 3068]" list=\
    bogons
add address=198.18.0.0/15 comment="NIDB Testing" list=bogons
add address=198.51.100.0/24 comment="Reserved - IANA - TestNet2" list=bogons
add address=203.0.113.0/24 comment="Reserved - IANA - TestNet3" list=bogons
add address=224.0.0.0/4 comment=\
    "MC, Class D, IANA # Check if you need this subnet before enable it" \
    disabled=yes list=bogons
/ip firewall filter
add action=drop chain=forward comment=\
    "Block standarised IP-ranges which should not be there" dst-address-list=\
    bogons in-interface-list=WAN
add action=accept chain=input comment=MNGT routing-mark=MNGT
add action=accept chain=output comment=MNGT routing-mark=MNGT
add action=accept chain=forward comment=MNGT routing-mark=MNGT
add action=accept chain=forward dst-address=192.168.218.18 log-prefix=16to18 \
    src-address=192.168.88.16
add action=accept chain=input log=yes log-prefix=in-src18 src-address=\
    192.168.218.18
add action=accept chain=input dst-address=192.168.218.18 log=yes log-prefix=\
    in-dst18
add action=accept chain=output log=yes log-prefix=out-src18 src-address=\
    192.168.218.18
add action=accept chain=output dst-address=192.168.218.18 log=yes log-prefix=\
    out-dst18
add action=accept chain=input comment=DEFAULT
add action=accept chain=output comment=DEFAULT
add action=accept chain=forward comment=DEFAULT connection-state="" \
    log-prefix=FORW
add action=drop chain=input log=yes
add action=drop chain=output log=yes
add action=drop chain=forward log=yes
/ip route
add comment="GW for MNGT" distance=1 gateway=MNGT-LAN routing-mark=MNGT
/ip service
set winbox disabled=yes
/routing bgp network
add comment="PC-LAN 10G" network=192.168.216.0/24
add comment="GreenZone 10G" network=192.168.218.0/24
/routing bgp peer
add name=pfSense remote-address=192.168.99.200 remote-as=65123 ttl=4
/system clock
set time-zone-name=Europe/Amsterdam
/system ntp client
set enabled=yes primary-ntp=192.168.10.1
/system routerboard settings
set boot-os=router-os
/system script
add comment="Generate Bogon List" dont-require-permissions=no name=\
    BogonScript owner=admin policy=\
    ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon source="/\
    ip firewall address-list\
    \nadd address=0.0.0.0/8 comment=\"Self-Identification [RFC 3330]\" disable\
    d=no list=bogons;\
    \nadd address=10.0.0.0/8 comment=\"Private[RFC 1918] - CLASS A # Check if \
    you need this subnet before enable it\"\\\
    \ndisabled=yes list=bogons;\
    \nadd address=127.0.0.0/8 comment=\"Loopback [RFC 3330]\" disabled=no list\
    =bogons;\
    \nadd address=169.254.0.0/16 comment=\"Link Local [RFC 3330]\" disabled=no\
    \_list=bogons;\
    \nadd address=172.16.0.0/12 comment=\"Private[RFC 1918] - CLASS B # Check \
    if you need this subnet before enable it\"\\\
    \ndisabled=yes list=bogons;\
    \nadd address=192.168.0.0/16 comment=\"Private[RFC 1918] - CLASS C # Check\
    \_if you need this subnet before enable it\"\\\
    \ndisabled=yes list=bogons;\
    \nadd address=192.0.2.0/24 comment=\"Reserved - IANA - TestNet1\" disabled\
    =no list=bogons;\
    \nadd address=192.88.99.0/24 comment=\"6to4 Relay Anycast [RFC 3068]\" dis\
    abled=no list=bogons;\
    \nadd address=198.18.0.0/15 comment=\"NIDB Testing\" disabled=no list=bogo\
    ns;\
    \nadd address=198.51.100.0/24 comment=\"Reserved - IANA - TestNet2\" disab\
    led=no list=bogons;\
    \nadd address=203.0.113.0/24 comment=\"Reserved - IANA - TestNet3\" disabl\
    ed=no list=bogons;\
    \nadd address=224.0.0.0/4 comment=\"MC, Class D, IANA # Check if you need \
    this subnet before enable it\"\\\
    \ndisabled=yes list=bogons;"
/system swos
set allow-from-ports=p5,p17 allow-from-vlan=88

mkx · Wed Sep 04, 2019 11:04 am

If I focus on VLAN 123, there are a few things that don't seem entirely right ... which might (or might not) explain why pinging from PC to pfSense (via CRS) doesn't work (but ARP, being L2 protocol, does):

VLAN interface VLAN123 should be parented to VirtualSwitch1 (not "04 RouterData" .. .because interface "04 RouterData" is member of said bridge)
setting pvid on bridge ports which haveframe-types=admit-only-vlan-tagged doesn't make much sense but shouldn't hurt either ... makes config less readable though
if you really want CRS to participate in VLAN 123 on L3 (creation of interface VLAN123 implies that), then "interface" VirtualSwitch1 should be tagged member of "bridge" VirtualSwitch1 (in section /interface bridge vlan). If that's not intended, then interace VLAN123 is not needed at all.

As I wrote: I can't find anything which would explain why L2 communication between PC and pfSense (via VLAN 123) somehow works, but L3 communication doesn't. Since CRS doesn't interact with this VLAN on L3, this doesn't even make any sense.

If I was you, I'd start off with blank configuration of CRS and would configure only what's really needed (i.g. only L2 stuff for VLAN 123) to see where's a problem. Only then I'd overload CRS with all the L3 stuff you have there. The fact is that CRS3xx perform quite well as switches apart from SFP+ link negotiation/stability that some users report on this forum. But if this was the problem in your case (log on CRS should reflect that), it would mess also with L2 (ARP) stuff.

Louis2 · Wed Sep 04, 2019 11:48 am

Hello,

Thanks for the replay. Here my findings:

VLAN interface VLAN123 should be parented to VirtualSwitch1 (not "04 RouterData" .. .because interface "04 RouterData" is member of said bridge)
>> and …… if that would be the case than it would be necessary to have the option to select a port !!??
>> what ever I tried, no effect

setting pvid on bridge ports which haveframe-types=admit-only-vlan-tagged doesn't make much sense but shouldn't hurt either ... makes config less readable though
>> agree, but the port is only temporarely in use for test purpose.

if you really want CRS to participate in VLAN 123 on L3 (creation of interface VLAN123 implies that), then "interface" VirtualSwitch1 should be tagged member of "bridge" VirtualSwitch1 (in section /interface bridge vlan). If that's not intended, then interace VLAN123 is not needed at all.
>> no, I do not want the bridge (if to be read as CPU) to participate, so I left it out. For the test I did include, did not make any difference

Note that the gui is very confusing to me:
- if an interface is part of the bridge, the bridge is allready involved ….. I would say
- so when to additionally include the bridge interface, if the bridge interface is really what the name suggest …. no idea
- however I think that the "bridge interface" is not the "bridge interferface" at all. I think it is the CPU interface ……

I did also expiriment with the switch menu adding the interfaces there also (1st at menu interfaces, 2e at menu bridge, 3e at menu switch ….), but that was not the solution as well, so I removed it

I also did remove the minimal bgp settings, since it was not active and, you never know, could change behavoir.

What ever, problem still there ……

Louis

mkx · Wed Sep 04, 2019 12:08 pm

Many people get confused because of bridge's dual personality:

"something like a switch" ... one defines member interfaces and bridge (more or less) intelligently moves traffic between those interfaces. In case of CRS3xx this is mostly offloaded to switch hardware, the rest of RouterBoard devices do this in general purpose CPU (in software)
"interface" ... which enables device's CPU to interact with network on L2 (and thus L3).
You can call it CPU interface if you will - but that might be counter-intuitive as one can have plenty of bridges and each comes with it's own interface ... so CPU might get plenty of interfaces ... on the other hand it is not possible to create purely software interface (a CPU interface), although it would come handy at times, the closest approximation is a bridge without any explicitly set member interfaces.

The problem is, that the personality #2 gets created implicitly when admin creates personality #1 and bears the same name. If creation was explicit (as is the case with, e.g., VLAN interfaces), this would be slightly more clear. The use, however, makes clear that the "split personality" is really going on ... whenever bridge's name is used in context where otherwise interfaces are used it's the personality #2 which is used.

If you accept this kind of view on bridge, then it becomes clear why bridge has to be tagged (or untagged) member of self if you want to make CPU interact with selected VLAN(s).

Again: I'd reset CRS to factory default setup and start building VLAN setup step-by-step.
There have been (rare) reports where devices misbehaved, but after they've been factory reset and exactly same config imported (via export/import script, not via backup/restore), seemingly identical config made them work as expected. Seems like too many reconfigurations might leave some (invisible) traces making devices misbehave.

Louis2 · Wed Sep 04, 2019 1:07 pm

Hello mkx,

Because of your suggestion and because I am desparate, I did reset the switch to defaults and did configure test VLAN from zero.

Conform your suggestion
- I did take the bridge as leading interface in the "interface menu" and
- I also included it as "tagged" in the "bridge VLAN-interface" and
- I tried with both the "VLAN-port" included and not included in the bridge VLAN-set-up (tagged)
- Appart from link negotiation speed, I did not change any thing in the default config!

Rather vague:
- if you should include the VLAN-port
- and when the bridge/cpu-port

What ever, it did not change the behavoir. Still can ping on mac-level, but can not ping on IP-level.

Below the actual config.

Sincerely,

Louis

# jan/02/1970 00:41:28 by RouterOS 6.45.5
# software id = UT7L-U4J9
#
# model = CRS317-1G-16S+
# serial number = xyz
/interface bridgeD:E0:
add admin-mac=xyz auto-mac=no comment=defconf name=bridge \
vlan-filtering=yes
/interface ethernet
set [ find default-name=ether1 ] advertise=100M-full,1000M-full name=\
MNGT-port
set [ find default-name=sfp-sfpplus1 ] advertise=100M-full,1000M-full name=\
Test-PC
set [ find default-name=sfp-sfpplus4 ] name=pfSense-Connection
set [ find default-name=sfp-sfpplus11 ] disabled=yes
set [ find default-name=sfp-sfpplus12 ] disabled=yes
set [ find default-name=sfp-sfpplus13 ] disabled=yes
/interface vlan
add interface=bridge name=VLAN123 vlan-id=123
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/interface bridge port
add bridge=bridge comment=defconf interface=MNGT-port
add bridge=bridge comment=defconf interface=Test-PC pvid=123
add bridge=bridge comment=defconf interface=sfp-sfpplus2
add bridge=bridge comment=defconf interface=sfp-sfpplus3
add bridge=bridge comment=defconf interface=pfSense-Connection
add bridge=bridge comment=defconf interface=sfp-sfpplus5
add bridge=bridge comment=defconf interface=sfp-sfpplus6
add bridge=bridge comment=defconf interface=sfp-sfpplus7
add bridge=bridge comment=defconf interface=sfp-sfpplus8
add bridge=bridge comment=defconf interface=sfp-sfpplus9
add bridge=bridge comment=defconf interface=sfp-sfpplus10
add bridge=bridge comment=defconf interface=sfp-sfpplus11
add bridge=bridge comment=defconf interface=sfp-sfpplus12
add bridge=bridge comment=defconf interface=sfp-sfpplus13
add bridge=bridge comment=defconf interface=sfp-sfpplus14
add bridge=bridge comment=defconf interface=sfp-sfpplus15
add bridge=bridge comment=defconf interface=sfp-sfpplus16
/interface bridge vlan
add bridge=bridge tagged=pfSense-Connection,bridge untagged=Test-PC vlan-ids=\
123
/ip address
add address=192.168.88.1/24 comment=defconf interface=bridge network=\
192.168.88.0
/system routerboard settings
set boot-os=router-os
/system swos
set allow-from-ports=p5,p17 allow-from-vlan=88

CZFan · Wed Sep 04, 2019 10:20 pm

If you are the trying to ping from the CRS device, then the IP address must be on vlan123 and not the bridge

Louis2 · Thu Sep 05, 2019 8:23 am

Halo,

Thanks for your suggestion. It is not the solution for my problem, however I never thought about the option to put an IP on a vlan. I will try later.

Two things:

1) The VLAN I would like to create is a transparrant VLAN. It should not have an IP-address. I need to reach/ping the real devices attached to (!) the VLAN, not a "VLAN-device" (whatever it is)

2) One of my frustrations is that, for as far as I know, it is not possible to assign an addres to a port or the CPU or the Bridge at all !!

That will say:
* you can, in a less transparant way, define the addres of e.g. the gateway. However you can not define the e.g. the addres of the CPU-port in a VLAN.
* as example: I have a management lan and via that managment lan I would like to reach the switch its gui. Nothing more!
(The Switch should NOT have the addres range, do routing or create a gateway or what ever)
* So I need a command to assign addres-x to (cpu-)port vlan-id-xy, without any further impact … not possible, I think

Louis

mkx · Thu Sep 05, 2019 11:18 am

1) The VLAN I would like to create is a transparrant VLAN. It should not have an IP-address. I need to reach/ping the real devices attached to (!) the VLAN, not a "VLAN-device" (whatever it is)

You wrote that you can get ARP communication between the two "real" devices ... kindly explain us what do you mean by that?

2) One of my frustrations is that, for as far as I know, it is not possible to assign an addres to a port or the CPU or the Bridge at all !!

I thought I explained that in my post #14 above. I obviously failed miserably.

That will say:
* you can, in a less transparant way, define the addres of e.g. the gateway. However you can not define the e.g. the addres of the CPU-port in a VLAN.

RouterOS by default will route between L3 interfaces where it has IP config. In your case only single interface has IP address defined (which is "interface" bridge ... doesn't work as @Sob noted though) and thus can not route any traffic anywhere else.
If CRS did have more than one L3 interface configured with IP address, then you would have to use /ip firewall to block unwanted forwarding ... a pretty simple rule would block it.
Either way, as routing is strictly L3 operation which is on Routerboards always performed in software by CPU, blocking in firewall doesn't affect switching performance (if done in switch chip as it should be done on CRS3xx). Neither does it affect bridging performance if CPU doesn't get overloaded due to amount of bridged+routed traffic, because IP firewall in principle doesn't affect bridged traffic.

Louis2 · Thu Sep 05, 2019 11:55 am

Reaction below.

1) The VLAN I would like to create is a transparrant VLAN. It should not have an IP-address. I need to reach/ping the real devices attached to (!) the VLAN, not a "VLAN-device" (whatever it is)
You wrote that you can get ARP communication between the two "real" devices ... kindly explain us what do you mean by that?

As explained the test VLAN uses two ports. one towards pfSense, the other towards the test-pc.
- pfsense has the gateway and the DHCP-server
- the CRS has no IP-adress or route (intentionally, it should just route level-2)

- If I connect the PC, it gets IPV4 and IPV6 addresses assigned from pfsense. So that works.
- If I try to ping pfsense with the RouterOS ping tool using standard ping, or try to do the same from the test-pc, I do not get a connection
- if I use the RouterOS ping tool with the arp option, I get answers. If I use a "arpping" tool on the PC I also gets the ping back

So there definitvely is a L2 connection, but for some stupid reason, the switch is not willing to pass IP.

So understand that what I want the swith is to provide a (level-2) VLAN ! (not a level 3 gateway)

That the CRS can do more, excelent, but not for this VLAN!

Louis

CZFan · Thu Sep 05, 2019 12:22 pm

Lets 1st setup management access for CRS (Using Vlan 123 as example)

/interface bridge
add fast-forward=no frame-types=admit-only-vlan-tagged ingress-filtering=yes name=bridge1 vlan-filtering=yes
/interface vlan
add interface=bridge1 name=MGMT vlan-id=123
/ip address
add address=10.1.2.3/24 interface=MGMT network=10.1.2.0
/interface bridge port
add bridge=bridge1 frame-types=admit-only-vlan-tagged ingress-filtering=yes interface=ether1
add bridge=bridge1 frame-types=admit-only-vlan-tagged ingress-filtering=yes interface=ether2
add bridge=bridge1 frame-types=admit-only-vlan-tagged ingress-filtering=yes interface=ether3
add bridge=bridge1 frame-types=admit-only-untagged-and-priority-tagged ingress-filtering=yes interface=ether4 pvid=123
/interface bridge vlan
add bridge=bridge1 tagged=bridge1,ether1,ether2,ether3 untagged=ether4 vlan-ids=123

The above will:
Create a bridge called "bridge1", add ports ether1 - 4 to this bridge, create a vlan with id 123, assign this vlan to the bridge and attach an IP to this vlan
It will also only allow only "tagged" frames for vlan id 123 on ports ether1-3, i.e. Trunk ports, and port ether4 is an Access port, i.e. ingress frames gets tagged as they come into the port based on pvid value, and frames matching this pvid value will also be stripped egress
The bridge "bridge1" must be part of "tagged" config in order to access the device / cpu port via this vlan

So now you create another vlan on pfsense (connected to ether1 on CRS), vlan id 321 and the client behind the CRS on ether5 will be normal workstations, i.e. they will not tag frames and need an "Access Port" then the vlan sub interface must exist on the CRS, so then you add

/interface vlan
add interface=bridge1 name=DATA vlan-id=321
/interface bride port
add bridge=bridge1 frame-types=admit-only-untagged-and-priority-tagged ingress-filtering=yes interface=ether5 pvid=321
/interface bridge vlan
add bridge=bridge1 tagged=ether1 untagged=ether5 vlan-ids=321

and voila, you have your "transparent" vlan, if it does not work, then your problem is on the pfsense side

If behind ether5 on CRS is a server, that will do tagging of frames, then you will not need a vlan sub interface on the CRS, and config will only be:

/interface bride port
add bridge=bridge1 frame-types=admit-only-vlan-tagged ingress-filtering=yes interface=ether5
/interface bridge vlan
add bridge=bridge1 tagged=ether1,ether5 vlan-ids=321

For more info, see:
https://wiki.mikrotik.com/wiki/Manual:I ... _Filtering

Louis2 · Thu Sep 05, 2019 2:26 pm

Hi,

Thanks for giving this example! I compared the proposed config which my config.

I compared my export file with your statements (in most cases I am using the gui)

Delta’s “interface vlan”
- No delta’s
Delta’s “ip address”
- You use the port as interface where
- I use the VLAN as interface
==> that is a significant delta; Question is why should you use what, interface, bridge or vlan !?

Delta’s “interface bridge port”
- You do not specify a PIVD with every interface where
- I specify an PIVD everywhere (I think in the gui it not even possible not to assign an PIVD)
==> I assume that not defining a PIVD implies that existing (default) PIVD stays.
Note that I mostly change that PIVD because I do not like the default PIVD-1 especially not since it creates some uncontrolled path between interfaces

Delta’s “interface vlan”
- No delta’s I assume a part of the added word “only” in next sentence
- The bridge "bridge1" must <only> be part of "tagged" config in order to access the device / cpu port via this vlan
 but I think we agree

Big question is “why is it not working!!??” In fact the fact that I do not understand, is perhaps even more frustrating than the fact that it is not working.
Note that I have been thinking in the direction of “xstp (rstp,mstp)” as problem cause (tried “none” for a moment), but IMHO also level-2 connectivity should not be there.

Whatever, I spend day’s on this problem and another couple of days on the fact, that I could not define assigning an IP-addres on the CPU to reach the CPU, without creating a VLAN-gateway on my coming from “external” VLAN. With lots of routing problems as consequence (traffic leaving the VLAN).

And then I did the router performance test “dramatic”. No other words (~ 250 mbit, with hardly FW-rules).

So yesterday evening I decided to switch to SwitchOS. Not what I had in mind(!), but guess what, in opposite to RouterOS it is working as expected within half a day!
(I really had expected the same from RouterOS. It really is a pity !!!).

I would love to go back to RouterOS, but simple things like the issues (vlan's, addres assignment, etc) described above, should work (!) and without any hazzle (!).
That is the situation for now.

Sincerely,
Louis
PS will probably try next RouterOS version if it comes available

mkx · Thu Sep 05, 2019 3:43 pm

- If I connect the PC, it gets IPV4 and IPV6 addresses assigned from pfsense. So that works.

Meaning that CRS is doing its job just fine.

- If I try to ping pfsense with the RouterOS ping tool using standard ping, or try to do the same from the test-pc, I do not get a connection
- if I use the RouterOS ping tool with the arp option, I get answers. If I use a "arpping" tool on the PC I also gets the ping back

RouterOS can not ping using IP pings as it doesn't have IP address ... and you made it very clear that that's the way you want to have it.

As to why client PC can't IP ping pfSense despite having L2 connectivity ... well that's a completely different question. You may want to debug things a little. Use wireshark on client PC to check the pings (do ICMP echo requests go out of port, do ICMP echo replies arrive back). If you see neither, then it's client PCs firewall blocking outgoing ICMP echo requests. If you can see both, then it's client PCs firewall blocking ICMP echo replies. If you can only see the outgoing, then it's something on either CRS (I highly doubt that) or on pfSense (it might block the ICMP itself).

You could try to configure CRS to mirror one of ports, connect another PC there and use wireshark to observe what's going on. If you instruct CRS to mirror both directions, then you should see both request and reply. If you will mirror port sfp-sfpplus4 and see the outgoing ICMP echo request, then you'll know that CRS is (highly probably) doing its job ...

Is there any other service on pfSense available to client PC? If yes, try to test using that service.

Louis2 · Fri Sep 06, 2019 12:34 pm

My excuses,

Of course I do agree with every one saying that “ping not possible” with “Level-2 OK”, is “very unlikely.

So I did repeat the test again, using my actual config based on SwitchOS. Same problem. So I did double check things.
And I found the problem. A type in the pfSense firewall rule which blocked the ping. So “1000 times sorry”.

Glad I found the issue

Sincerely,

Louis

Louis2 · Fri Sep 06, 2019 2:55 pm

For information, I would like to make an additional remark.

The main reason that I did not verdict the firewall, is that I could not ping the subnets gateway. My expectation was that you can always reach / ping the gateway, independent from whatever the FW-rules are. The gateway is after all part of the involved subnet. That turned out to be wrong. At least in pfSense (probably on other FW’s as well), you cannot ping the GW, it there is no FW-rule allowing that.

Louis

mkx · Fri Sep 06, 2019 3:09 pm

At least in pfSense (probably on other FW’s as well), you cannot ping the GW, it there is no FW-rule allowing that.

Depends how particular FW philosophy works. In ROS, default behaviour is that FW allows everything. One can revert this by explicit drop as last rule in rule chain.

At the same time, ROS has only one input chain, which filters communication with the device itself, and doesn't matter which IP address is targeted ... unless FW rule is explicitly using this information as criterion. Which means that if you're pinging router, it will (most probably) answer to pings targeting any of it's IP addresses / interfaces even if certain subnet is not accessible to the pinging client.

Cannot get simple VLAN to work on CRS317 ☹

Cannot get simple VLAN to work on CRS317 ☹

Re: Cannot get simple VLAN to work on CRS317 ☹

Re: Cannot get simple VLAN to work on CRS317 ☹

Re: Cannot get simple VLAN to work on CRS317 ☹

Re: Cannot get simple VLAN to work on CRS317 ☹

Re: Cannot get simple VLAN to work on CRS317 ☹

Re: Cannot get simple VLAN to work on CRS317 ☹

Re: Cannot get simple VLAN to work on CRS317 ☹

Re: Cannot get simple VLAN to work on CRS317 ☹

Re: Cannot get simple VLAN to work on CRS317 ☹

Re: Cannot get simple VLAN to work on CRS317 ☹

Re: Cannot get simple VLAN to work on CRS317 ☹

Re: Cannot get simple VLAN to work on CRS317 ☹

Re: Cannot get simple VLAN to work on CRS317 ☹

Re: Cannot get simple VLAN to work on CRS317 ☹

Re: Cannot get simple VLAN to work on CRS317 ☹

Re: Cannot get simple VLAN to work on CRS317 ☹

Re: Cannot get simple VLAN to work on CRS317 ☹

Re: Cannot get simple VLAN to work on CRS317 ☹

Re: Cannot get simple VLAN to work on CRS317 ☹

Re: Cannot get simple VLAN to work on CRS317 ☹

Re: Cannot get simple VLAN to work on CRS317 ☹

Re: Cannot get simple VLAN to work on CRS317 ☹

Re: Cannot get simple VLAN to work on CRS317 ☹

Re: Cannot get simple VLAN to work on CRS317 ☹

Who is online