Community discussions

 
peson
Trainer
Trainer
Posts: 177
Joined: Tue Jul 20, 2004 10:33 am
Location: Sweden

Re: QinQ VLAN's Help needed

Tue Apr 16, 2019 8:51 am

Also on advice of MT support I decided to go with CVID tag stacking instead of c-vlan within s-vlan.

I build following config:
/interface bridge
add ingress-filtering=yes name=bridge vlan-filtering=yes

/interface vlan
add interface=sfp-sfpplus1 name=vlan-gs-ser vlan-id=309
add interface=bridge name=vlan-mgmt vlan-id=20

/interface bridge port
add bridge=bridge ingress-filtering=yes interface=vlan-gs-ser tag-stacking=yes
add bridge=bridge frame-types=admit-only-vlan-tagged ingress-filtering=yes interface=sfp-sfpplus1

/interface bridge vlan
add bridge=bridge comment=transport-gs-ser tagged=bridge,sfp-sfpplus1,vlan-gs-ser vlan-ids=309
add bridge=bridge comment=mgmt tagged=bridge,vlan-gs-ser,sfp-sfpplus6,sfp-sfpplus7,sfp-sfpplus8,trunk-ccr2-crs2 vlan-ids= 20
Without tag-stacking option on the vlan-gs-ser port everything seem to work fine, but as soon as I enable tag stacking it got broken. Anyone got an idea?
I think you missundersstand the tag-stacking feature.
From the example on page: https://wiki.mikrotik.com/wiki/Manual:B ... g_Stacking
"What we want to achieve is that regardless what is being received on ether2 and ether3, a new VLAN tag will be added to encapsulate the traffic that is coming from those ports. What tag-stacking does is forces a new VLAN tag, so we can use this property to achieve our desired setup. We are going to be using the same configuration as in the Trunk/Access port setup, but with tag stacking enabled on the access ports:"
In your example, if 309 is the outer tag and sfp-sfpplus1 is the port to metro provider, it would be something like:
/interface bridge
add name=bridge vlan-filtering=yes ether-type=0x8100
/interface bridge port
add bridge=bridge interface=sfp-sfpplus1
add bridge=bridge interface=sfp-sfpplus6 tag-stacking=yes pvid=309 (outer tag)
add bridge=bridge interface=sfp-sfpplus7 tag-stacking=yes pvid=309 (outer tag)
add bridge=bridge interface=sfp-sfpplus8 tag-stacking=yes pvid=309 (outer tag)
...
/interface bridge vlan
add bridge=bridge tagged=sfp-sfpplus1 untagged=sfp-sfpplus6,sfp-sfpplus7,sfp-sfpplus8 vlan-ids=309
In "/interface vlan" you only add the VLAN that needs to be processed by the CPU, not the switch chip.
 
deepmedia
just joined
Posts: 15
Joined: Sat Dec 29, 2018 4:19 pm

Re: QinQ VLAN's Help needed

Tue Apr 16, 2019 11:28 am

The main reason I didnt use pvid 309 on the endports is the fact we have untagged traffic that needs both an inner and outer vlan tag. Is there a way to double tag them? Or do you suggest to always tag all traffic on the Customers side?
 
sindy
Forum Guru
Forum Guru
Posts: 3287
Joined: Mon Dec 04, 2017 9:19 pm

Re: QinQ VLAN's Help needed

Tue Apr 16, 2019 1:30 pm

That's what I've tried to explain before in post #49 - it makes no difference whether you place an S-tag over an existing C-tag or whether you nest two C-tags, the real problem is that only a single tag can be added to a frame at ingress and none at egress, and just a single tag can be removed from a frame at egress and none at ingress. So when a tagless frame comes in through ether2, you can tag it with C-VID 10, but all the way to the wire connected to ether4 there is no place where you could add the other tag with S-VID or C-VID 100, unless you cascade two bridges as in my drawing.

If you can make all the customers send you only C-tagged frames, you'll be in a better position because you'll only need to add the outer tags, so in this case tag-stacking=yes will ensure that the existing C-tags of the ingress frames will be ignored and new C-tags will be added on ports configured as "untagged". But if you set the ether-type=0x88a8 on the bridge, you don't even need the tag-stacking=yes setting because the C-tags (0x8100) of ingress frames will be ignored because the bridge won't recognize frames with ethertype 0x8100 as tagged ones.
Instead of writing novels, post /export hide-sensitive. Use find&replace in your favourite text editor to systematically replace all occurrences of each public IP address potentially identifying you by a distinctive pattern such as my.public.ip.1.
 
deepmedia
just joined
Posts: 15
Joined: Sat Dec 29, 2018 4:19 pm

Re: QinQ VLAN's Help needed

Tue Apr 16, 2019 1:36 pm

Sindy, that's exactly what I was already thinking. 1st of all I will start by making all my traffic towards and from our customers tagged. We'll see if that will work out. Thanks!

Verstuurd vanaf mijn Pixel 3 met Tapatalk

 
deepmedia
just joined
Posts: 15
Joined: Sat Dec 29, 2018 4:19 pm

Re: QinQ VLAN's Help needed

Fri Jun 07, 2019 11:49 pm

To get a workaround I bought 2 additional CRS317's (I needed them anyway)

I built the following setup:
Untitled.png

The upper CRS is configured using the following config:
/interface bridge
add ingress-filtering=yes name=bridge vlan-filtering=yes
/interface ethernet
set [ find default-name=ether1 ] speed=100Mbps
set [ find default-name=sfp-sfpplus1 ] speed=10Gbps
set [ find default-name=sfp-sfpplus16 ] speed=10Gbps
/interface bridge port
add bridge=bridge ingress-filtering=yes interface=sfp-sfpplus1 pvid=309 tag-stacking=yes
add bridge=bridge frame-types=admit-only-vlan-tagged ingress-filtering=yes interface=sfp-sfpplus16
add bridge=bridge ingress-filtering=yes interface=ether1
/interface bridge vlan
add bridge=bridge tagged=bridge,sfp-sfpplus16,ether1 vlan-ids=80
add bridge=bridge tagged=bridge untagged=sfp-sfpplus1 vlan-ids=10

The lower CRS got the following config::
/interface bridge
add ingress-filtering=yes name=bridge vlan-filtering=yes
/interface ethernet
set [ find default-name=ether1 ] speed=100Mbps
set [ find default-name=sfp-sfpplus1 ] speed=10Gbps
set [ find default-name=sfp-sfpplus16 ] speed=10Gbps
/interface vlan
add interface=bridge name=vlan10 vlan-id=10
add interface=bridge name=vlan80 vlan-id=80
/interface bridge port
add bridge=bridge frame-types=admit-only-vlan-tagged ingress-filtering=yes interface=sfp-sfpplus1
add bridge=bridge frame-types=admit-only-vlan-tagged ingress-filtering=yes interface=sfp-sfpplus16
/interface bridge vlan
add bridge=bridge tagged=bridge,sfp-sfpplus1 vlan-ids=10
add bridge=bridge tagged=bridge,sfp-sfpplus16 vlan-ids=80
/ip address
add address=1.1.1.1/24 interface=vlan10 network=1.1.1.0
add address=2.2.2.2/24 interface=vlan80 network=2.2.2.0

Whenever all cables are connected only the double tagged traffic gets forwarded to the wireshark (see image)
allportsenabled.png

When disconnect sfp1 cable the single tagged traffic gets forwarded (see image)
disconnectcablesfp1.png

Is there any workaround you guys might think of? The only requirement is the fact that single and double tagged traffic needs to be on the same cable (because of the carrier who is providing these vlans)

Thanks for your suggestions.
You do not have the required permissions to view the files attached to this post.
 
sindy
Forum Guru
Forum Guru
Posts: 3287
Joined: Mon Dec 04, 2017 9:19 pm

Re: QinQ VLAN's Help needed

Sat Jun 08, 2019 12:29 am

I'd have to see your setup hands-on (but I don't plan to drive by any time soon), but I think you are missing one basic thing about L2 - in addition to VLAN-tagged traffic, there are also VLAN-agnostic protocols running on switches, among which STP is the one which causes this issue. Whenever you connect two L2 devices together by more than a single logical link (a bond of several physical links behaves as a single logical link), you have to address the omission of the L2 protocol designers which is the absence of any TTL field in the L2 header. In another words, you have to prevent loops in the logical topology by dynamically disabling all logical links except one. So although you only permit one VLAN on the physical link between sfpplus1 and only permit another VLAN on the physical link between sfpplus16, the RSTP protocol which runs on the bridges by default still only permits one of these links to be active at a time whenever both are physically connected.

So setting protocol-mode at both bridges (upper and lower) to none might resolve your issue, but it may also cause a broadcast storm if eventually some frames leak the wrong way and loop back through the other link. So if this happens, you'll have to set up also split horizon, preventing any frame received on sfpplus1 from being sent out via sfpplus16 and vice versa, whereas ether4 on the upper CRS and any port except sfpplus1 and sfpplus16 on the lower CRS will be able to forward frames to/from both sfpplus1 and sfpplus16.

Other than that, I'm afraid that in order to make it work in both directions, the second row in /interface bridge vlan in the upper CRS configuration has to be changed to add bridge=bridge tagged=bridge untagged=sfp-sfpplus1 vlan-ids=309. The pvid parameter of /interface bridge port row controls the ingress handling; the position of the interface on the tagged or untagged list on the vlan-ids row in /interface bridge vlan controls the egress handling. For the upper CRS, VLAN 10 doesn't exist at all, it only knows about VLAN 309 and VLAN 80.
Instead of writing novels, post /export hide-sensitive. Use find&replace in your favourite text editor to systematically replace all occurrences of each public IP address potentially identifying you by a distinctive pattern such as my.public.ip.1.
 
deepmedia
just joined
Posts: 15
Joined: Sat Dec 29, 2018 4:19 pm

Re: QinQ VLAN's Help needed

Sat Jun 08, 2019 8:27 am

Your suggestion turning off RSTP works... see image! Allthough I'm very excited to implement this on the network, on the other hand I am a little scared for broadcast storms as you suggested. Is there a way to verify / monitor this? And if this is the case, how to prevent it?

afterrstp-none.png

Oh and btw, In my previous post the last rule of the following config fell off by copy pasting, so current config about vlan statements is:
/interface bridge vlan
add bridge=bridge tagged=bridge,sfp-sfpplus16,ether1 vlan-ids=80
add bridge=bridge tagged=bridge untagged=sfp-sfpplus1 vlan-ids=10
add bridge=bridge tagged=bridge,ether1 vlan-ids=309
You do not have the required permissions to view the files attached to this post.
 
sindy
Forum Guru
Forum Guru
Posts: 3287
Joined: Mon Dec 04, 2017 9:19 pm

Re: QinQ VLAN's Help needed

Sat Jun 08, 2019 12:06 pm

Your suggestion turning off RSTP works... see image! Allthough I'm very excited to implement this on the network, on the other hand I am a little scared for broadcast storms as you suggested. Is there a way to verify / monitor this? And if this is the case, how to prevent it?
Better to prevent it than to monitor :) Check this.

Oh and btw, In my previous post the last rule of the following config fell off by copy pasting, so current config about vlan statements is:
/interface bridge vlan
add bridge=bridge tagged=bridge,sfp-sfpplus16,ether1 vlan-ids=80
add bridge=bridge tagged=bridge untagged=sfp-sfpplus1 vlan-ids=10
add bridge=bridge tagged=bridge,ether1 vlan-ids=309
Replace the last two lines above by the single one I've suggested - add bridge=bridge tagged=bridge,ether1 untagged=sfp-sfpplus1 vlan-ids=309. As said, VLAN 10 is invisible to the upper CRS, it is just a payload inside VLAN 309 like any other.
Instead of writing novels, post /export hide-sensitive. Use find&replace in your favourite text editor to systematically replace all occurrences of each public IP address potentially identifying you by a distinctive pattern such as my.public.ip.1.
 
deepmedia
just joined
Posts: 15
Joined: Sat Dec 29, 2018 4:19 pm

Re: QinQ VLAN's Help needed

Sat Jun 08, 2019 12:43 pm

In the configuration-to-be all ports will be added to a single bridge, with the protocol-mode set to none. No other bridges, no tagless or untagged (except for the stacked ones) will be accepted onto the switch. I am probably safe when all connections are managed and connected by myself :) By the way: it servers as a core/metro switch to connect the other datacentre to the primary one.

Current config of the upper one:
/interface bridge vlan
add bridge=bridge tagged=bridge,sfp-sfpplus16,ether1 vlan-ids=80
add bridge=bridge tagged=bridge,ether1 untagged=sfp-sfpplus1 vlan-ids=309
Still working flawless. No loops (traffic still 0bps and cpu ilde)

Thanks for your support, appreciated greatly!
 
sindy
Forum Guru
Forum Guru
Posts: 3287
Joined: Mon Dec 04, 2017 9:19 pm

Re: QinQ VLAN's Help needed

Sat Jun 08, 2019 3:13 pm

No other bridges, no tagless or untagged (except for the stacked ones) will be accepted onto the switch. I am probably safe when all connections are managed and connected by myself :)
Well, I'm not really sure how exactly the switch chip handles ingress frames which are not Ethernet II but 802.2 (i.e. the first two bytes following the MAC addresses represent frame size, not content type), i.e. whether it deems them "untagged" or rather "neither tagged nor untagged". And there are not just STP frames, some switch vendors use their proprietary frames for loop detection, which may take the long path provider_switch -> upperCRS.ether4 -> upperCRS.sfpplus1 -> lowerCRS.sfpplus1 -> lowerCRS.sfpplus16 -> upperCRS.sfpplus16 -> upperCRS.ether4 -> provider_switch and make the provider switch shut down the port. In worse case, the adjacent switch would not detect a loop and disable the port but frames which escape the ingress filtering would circulate there forever (as they have no TTL field to be used to count hops), gradually seizing all the bandwidth. Candidates are not just broadcast/multicast frames but also frames towards unicast MAC addresses from which an ingress frame never comes so the switch cannot associate them with a port.

So to prevent this from happening, the port isolation needs to be set at the more distant switch from the source (so against loop detection/eternally circulating frames coming from the provider switch, the lowerCRS needs to have sfpplus1 and sfpplus16 isolated from each other; against loop detection/eternally circulating frames coming from the customer switch, the upperCRS needs to have sfpplus1 and sfpplus16 isolated from each other).
Instead of writing novels, post /export hide-sensitive. Use find&replace in your favourite text editor to systematically replace all occurrences of each public IP address potentially identifying you by a distinctive pattern such as my.public.ip.1.
 
User avatar
CZFan
Forum Guru
Forum Guru
Topic Author
Posts: 1229
Joined: Sun Oct 09, 2016 8:25 pm
Location: South Africa, Randburg
Contact:

Re: QinQ VLAN's Help needed

Sat Jun 08, 2019 8:20 pm

@deepmedia
As a side note, I assume the 1.1.1.1, etc addresses are loopback addresses, anyway, personally I will stay away from them as they are routable on internet
MTCNA, MTCTCE, MTCRE & MTCINE
 
deepmedia
just joined
Posts: 15
Joined: Sat Dec 29, 2018 4:19 pm

Re: QinQ VLAN's Help needed

Thu Jun 13, 2019 4:07 pm

Those we're used in a disconnected environment, no worries :)

Who is online

Users browsing this forum: Bing [Bot] and 22 guests