Community discussions

MikroTik App
 
nscheffer
newbie
Topic Author
Posts: 45
Joined: Sun Mar 29, 2020 12:52 am

Transparent hEX S to change vlan-priority for DHCP request only

Sat Apr 03, 2021 1:25 pm

Hi,

I got two internet fiber access in with a provider and I would like to remove their box and use my own Firewall.
Actually my firewall cannot change vlan priority for a paquet send by himself (dhcpv4 request and dhcpv6-pd solicit).
I would like to insert in a transparent way a Mikrotik hEX S on each fiber with the following :

Fiber -> SFP ONT Nokia -> sfp1 hEX S -> ether3 hEX S -> Firewall
ether1 hEX S -> Mikrotik switch with PoE for management

I got two bridge :
- default all port except sfp1 and ether3
- orange_pro sfp1 and ether3

Before I try to use only a Mikrotik router with SFP ONT (with or without TP-LINK MC220) to access internet and I add the following filter or switch rule (depending of the Mikrotik used for test) :

with a bridge filter :
/interface vlan
add interface=ether1 name=vlan832 vlan-id=832

/interface bridge filter
add action=set-priority chain=output dst-port=67 ip-protocol=udp log=yes \
log-prefix="Set CoS6 on DHCP request" mac-protocol=ip new-priority=6 \
add action=set-priority chain=output dst-port=547 ip-protocol=udp log=yes \
log-prefix="Set CoS6 on DHCPv6-PD solicit" mac-protocol=ipv6 \
new-priority=6 out-interface=vlan832 passthrough=yes
/interface bridge port
add bridge=bridge-wan interface=vlan832

with a switch rule :
/interface vlan
add interface=ether1 name=vlan832 vlan-id=832

/interface ethernet switch rule
add dst-port=67 mac-protocol=ip new-vlan-priority=6 ports=switch1-cpu \
protocol=udp src-mac-address=74:4D:28:6D:F7:AB/FF:FF:FF:FF:FF:FF switch=\
switch1
add dst-mac-address=33:33:00:01:00:02/74:4D:28:6D:F7:AB mac-protocol=ipv6 \
new-vlan-priority=6 ports=switch1-cpu src-mac-address=\
74:4D:28:6D:F7:AB/FF:FF:FF:FF:FF:FF switch=switch1


How to do the same for DHCPv4 request initiated by the Firewall on vlan=832 trough a MikroTik hEX S on port ether3 who has a SFP ONT on port sfp1 and needs to change the vlan-priority to 6 only for this DHCPv4 request to be able to mount the internet access on the firewall for IPv4 ? The Mikrotik will be managed using the other ports and will be acting as a transparent router.
How to do the same for DHCPv6-PD solicit initiated by the Firewall on vlan=832 trough a MikroTik hEX S on port ether3 who has a SFP ONT on port sfp1 and needs to change the vlan-priority to 6 only for this DHCPv6 solicit to be able to mount the internet access on the firewall for IPv6 ? The Mikrotik will be managed using the other ports and will be acting as a transparent router.

Many thanks in advance.
Regards,
Nicolas
 
nscheffer
newbie
Topic Author
Posts: 45
Joined: Sun Mar 29, 2020 12:52 am

Re: Transparent hEX S to change vlan-priority for DHCP request only

Sun Apr 04, 2021 10:21 pm

Nobody knows how to do this ?
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Transparent hEX S to change vlan-priority for DHCP request only

Mon Apr 05, 2021 12:55 pm

It's almost the same except that in the bridge filter rules, you have to use chain=forward rather than chain=output, and add an in-interface=ether3 match condition. And you still have to use a dedicated bridge for VLAN 832, because the bridge filter rules currently do not support matching on IP headers of packets carried by VLAN-tagged frames, so the frame carrying the DHCP/PD packet must get tagged after passing through the bridge. So ether3 <-U-> bridge832 <-U-> interface vlan 832 <=T=> sfp1.

Depending on the bandwidth you get from the ISP, you may find the hEX S to be a bottleneck.

In /interface ethernet switch rules, you would have to attach the rule to ether3 rather than to switch1-cpu, but this is not relevant to hEX S as switch rules are not supported there.
 
nscheffer
newbie
Topic Author
Posts: 45
Joined: Sun Mar 29, 2020 12:52 am

Re: Transparent hEX S to change vlan-priority for DHCP request only

Mon Apr 05, 2021 2:26 pm

Thanks for the answer, seems to be clear for me.
My bridge filter will be applied between ether3 and the bridge using forward chain and got my vlan832...

If I got switch rule with a compatible hardware it's the same where I need to apply the switch rule instead a bridge filter ?

Agree that the hEX S will be the bottleneck and switch rules are not supported for vlan and also for sfp1. It's mainly to do a technical validation and then decide how to tune the setup to expect to match maximum bandwidth available...

I got 2 fiber access and would like to replace both box, one is 2Gbps/600Mbps and the other 1Gbps/1Gbps.
I plan to replace my hEX S for each box by :
- RB2011iLS-IN
- or RB935GS-5HnT-RP

I will be able to use switch rule on the sfp port !
What is your advise for best Mikrotik product to fit this needs as a transparent sfp/ether CoS tagging device ?
What is your advise is I would like to have a sfp+ port instead ? (SFP ONT used is a Nokia G-010S-A with HSGMII at 2,5Gbps to be able to use the 2Gbps available on the fiber contract)

Thanks in advance.
Regards
Nicolas
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Transparent hEX S to change vlan-priority for DHCP request only

Mon Apr 05, 2021 3:35 pm

Agree that the hEX S will be the bottleneck and switch rules are not supported for vlan and also for sfp1.
On MT7621 based routers, RouterOS doesn't support switch chip rules at all, not only on sfp1.

I plan to replace my hEX S for each box by :
- RB2011iLS-IN
- or RB935GS-5HnT-RP
Both should work fine at wire speed if you use switch chip rules.

I got 2 fiber access and would like to replace both box, one is 2Gbps/600Mbps and the other 1Gbps/1Gbps.
...
What is your advise for best Mikrotik product to fit this needs as a transparent sfp/ether CoS tagging device ?
What is your advise is I would like to have a sfp+ port instead ? (SFP ONT used is a Nokia G-010S-A with HSGMII at 2,5Gbps to be able to use the 2Gbps available on the fiber contract)
According to the product information, CRS305-1G-4S+IN seems to be the best choice for your special purpose - one SFP+ towards the firewall, plain SFP towards the slower WAN, and another SFP+ towards the 2G/600M WAN if the firewall supports VLANs. But I've got no hands-on experience with this device. Don't forget to disable xSTP before connecting it to the ISPs.
 
nscheffer
newbie
Topic Author
Posts: 45
Joined: Sun Mar 29, 2020 12:52 am

Re: Transparent hEX S to change vlan-priority for DHCP request only

Mon Apr 05, 2021 3:45 pm

Sindy,

Thanks a lot for the first part of answer.
Concerning the CRS305, agree, seems to be the perfect match for me and I was thinking to use it for both fiber isolating each dual group of ports needed (SFP ONT into sfp1 and sfp2 going to firewall for Fiber 1 and Fiber 2 with sfp3 and sfp4 for example) and using the ethernet port for management...
I bought one 10 days ago and try to insert SFP ONT into one port, got nothing !!! SFP ONT invisible for the Mikrotik.. Try with 3 different SFP ONT and on each port same problem, so I decided to ship it back !
May be I made a mistake ? I miss something to have SFP ONT working ?
 
User avatar
mkx
Forum Guru
Forum Guru
Posts: 11381
Joined: Thu Mar 03, 2016 10:23 pm

Re: Transparent hEX S to change vlan-priority for DHCP request only

Mon Apr 05, 2021 3:59 pm

Mikrotik support for ONT SFPs is non existent so some might work and most don't. Even compatibility with "normal" SFPs is incomplete (mildly put). Which means that trying to get ONT SFP to work with any MT device is similar to trying to win a jackpot, even if particular ONT SFP works with one model of MT, it might not with another model of MT.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: Transparent hEX S to change vlan-priority for DHCP request only

Mon Apr 05, 2021 5:32 pm

Mikrotik support for ONT SFPs is non existent so some might work and most don't. Even compatibility with "normal" SFPs is incomplete (mildly put). Which means that trying to get ONT SFP to work with any MT device is similar to trying to win a jackpot, even if particular ONT SFP works with one model of MT, it might not with another model of MT.
So much for standards sigh. :-(
 
User avatar
mkx
Forum Guru
Forum Guru
Posts: 11381
Joined: Thu Mar 03, 2016 10:23 pm

Re: Transparent hEX S to change vlan-priority for DHCP request only

Mon Apr 05, 2021 6:33 pm

Standards ... one thing is to support normal SFPs which (semi-)transparently pass bits between left and right. And it's a pitty these are not more compatible. Which mostly is not result of poor standards but rather bad practice by major players who introduced incompatible extensions.

The other problem is support for intelligent SFP devices, such as ONTs. In this case SFP devices implement (optional) interfaces and not every SFP "chassy" supports those. It's somehow similar to SD cards where most of cards implement storage functionality, but SD can do more, like WiFi ... but not every SD card reader supports the less common functions.
 
nscheffer
newbie
Topic Author
Posts: 45
Joined: Sun Mar 29, 2020 12:52 am

Re: Transparent hEX S to change vlan-priority for DHCP request only

Mon Apr 05, 2021 7:00 pm

Agree that the hEX S will be the bottleneck and switch rules are not supported for vlan and also for sfp1.
On MT7621 based routers, RouterOS doesn't support switch chip rules at all, not only on sfp1.

I plan to replace my hEX S for each box by :
- RB2011iLS-IN
- or RB935GS-5HnT-RP
Both should work fine at wire speed if you use switch chip rules.
Sindy,

I just order a RB2011 to do my test.
If I am correct I will do :
- a new bridge including the sfp port and one Gig Ethernet port
- vlan 832 part of this bridge
- a simple switch rule to set the new vlan priority for just DHCP request

That's all correct ?
I hope also that my SFP ONT will be recognize on the new RB2011...
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Transparent hEX S to change vlan-priority for DHCP request only

Mon Apr 05, 2021 8:46 pm

As you want it wirespeed, the switch chip must do all the job. Hence you have to make sfp1 and etherX member ports of a bridge with hw=yes on the respective /interface bridge port rows, and if you ever add any other port of that switch chip to any other bridge, it must be added with hw=no (only a single bridge can be "hardware accelerated" per switch chip, and RouterOS chooses autonomously if you permit hw=yes on multiple bridges).

If I remember it properly, you don't need to configure VLAN 832 handling in the bridge configuration, just set

/interface ethernet switch port set etherX vlan-mode=secure default-vlan-id=832
/interface ethernet switch port set sfp1 vlan-mode=secure


/interface ethernet switch vlan add ports=etherX,sfp1 switch=switch1 vlan-id=832

/interface ethernet switch rule add dst-port=67 new-vlan-priority=6 ports=etherX protocol=udp switch=switch1 vlan-id=832

In the unlikely case that the VLAN tag is added on ingress after the frame has been processed by the rules (I'm not 100 % sure here and can't test right now), remove the vlan-id=832 from the match conditions of the rule. But in your case matching on vlan-id doesn't matter much as no other VLANs will be used on the switch chip anyway.
 
nscheffer
newbie
Topic Author
Posts: 45
Joined: Sun Mar 29, 2020 12:52 am

Re: Transparent hEX S to change vlan-priority for DHCP request only

Mon Apr 05, 2021 8:49 pm

As you want it wirespeed, the switch chip must do all the job. Hence you have to make sfp1 and etherX member ports of a bridge with hw=yes on the respective /interface bridge port rows, and if you ever add any other port of that switch chip to any other bridge, it must be added with hw=no (only a single bridge can be "hardware accelerated" per switch chip, and RouterOS chooses autonomously if you permit hw=yes on multiple bridges).

If I remember it properly, you don't need to configure VLAN 832 handling in the bridge configuration, just set

/interface ethernet switch port set etherX vlan-mode=secure default-vlan-id=832
/interface ethernet switch port set sfp1 vlan-mode=secure


/interface ethernet switch vlan add ports=etherX,sfp1 switch=switch1 vlan-id=832

/interface ethernet switch rule add dst-port=67 new-vlan-priority=6 ports=etherX protocol=udp switch=switch1 vlan-id=832

In the unlikely case that the VLAN tag is added on ingress after the frame has been processed by the rules (I'm not 100 % sure here and can't test right now), remove the vlan-id=832 from the match conditions of the rule. But in your case matching on vlan-id doesn't matter much as no other VLANs will be used on the switch chip anyway.
Hi Sindy,
Great and thanks for quick answer !
I should receive the new RB2011 in 2 days from now, I let you know when first test is done !
 
nscheffer
newbie
Topic Author
Posts: 45
Joined: Sun Mar 29, 2020 12:52 am

Re: Transparent hEX S to change vlan-priority for DHCP request only

Wed Apr 07, 2021 8:53 pm

As you want it wirespeed, the switch chip must do all the job. Hence you have to make sfp1 and etherX member ports of a bridge with hw=yes on the respective /interface bridge port rows, and if you ever add any other port of that switch chip to any other bridge, it must be added with hw=no (only a single bridge can be "hardware accelerated" per switch chip, and RouterOS chooses autonomously if you permit hw=yes on multiple bridges).

If I remember it properly, you don't need to configure VLAN 832 handling in the bridge configuration, just set

/interface ethernet switch port set etherX vlan-mode=secure default-vlan-id=832
/interface ethernet switch port set sfp1 vlan-mode=secure


/interface ethernet switch vlan add ports=etherX,sfp1 switch=switch1 vlan-id=832

/interface ethernet switch rule add dst-port=67 new-vlan-priority=6 ports=etherX protocol=udp switch=switch1 vlan-id=832

In the unlikely case that the VLAN tag is added on ingress after the frame has been processed by the rules (I'm not 100 % sure here and can't test right now), remove the vlan-id=832 from the match conditions of the rule. But in your case matching on vlan-id doesn't matter much as no other VLANs will be used on the switch chip anyway.
Hi Sindy,
I just receive the RB2011 today !
I try quickly but not working yet. I decide to check and to add a mirror switch rule form sfp1 to ether5 to capture trafic. But I don't know if the main switch rule to change CoS is still enabled ?
My firewall is on ether3 and the SFP ONT into sfp1.
I am connected on port ether6 for management.

Here is my config :

/interface bridge
add admin-mac=08:55:31:B8:F1:E0 auto-mac=no name=bridge
add name=orange-pro
/interface ethernet switch
set 0 mirror-source=sfp1 mirror-target=ether5
/interface ethernet switch port
set 0 vlan-mode=secure
set 3 default-vlan-id=832 vlan-mode=secure
/interface bridge port
add bridge=orange-pro interface=ether2
add bridge=orange-pro interface=ether3
add bridge=orange-pro interface=ether4
add bridge=orange-pro interface=ether5
add bridge=bridge interface=ether6
add bridge=bridge interface=ether7
add bridge=bridge interface=ether8
add bridge=bridge interface=ether9
add bridge=bridge interface=ether10
add bridge=orange-pro interface=sfp1
add bridge=orange-pro interface=ether1
/ip neighbor discovery-settings
set discover-interface-list=*2000011
/interface ethernet switch rule
add comment="Set CoS 6 DHCPv4 Request" dst-port=67 mac-protocol=ip new-vlan-priority=6 ports=ether3 protocol=udp src-port=68 \
switch=switch1 vlan-id=832
/interface ethernet switch vlan
add independent-learning=yes ports=ether3,sfp1 switch=switch1 vlan-id=832
/ip dhcp-client
add disabled=no interface=bridge
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Transparent hEX S to change vlan-priority for DHCP request only

Wed Apr 07, 2021 9:24 pm

I have a problem with mac-protocol=ip in the switch chip rule, because mac-protocol is vlan here if tagging is done before the rule is applied.

In bridge filter rules, the IP and port matching is only possible when mac-protocol=ip, but if I remember correctly, this is not the case with switch rules.
 
nscheffer
newbie
Topic Author
Posts: 45
Joined: Sun Mar 29, 2020 12:52 am

Re: Transparent hEX S to change vlan-priority for DHCP request only

Wed Apr 07, 2021 10:55 pm

I have a problem with mac-protocol=ip in the switch chip rule, because mac-protocol is vlan here if tagging is done before the rule is applied.

In bridge filter rules, the IP and port matching is only possible when mac-protocol=ip, but if I remember correctly, this is not the case with switch rules.
Hi Sindy,

When I was using in the past a Mikrotik hEX PoE to replace my box using an ONT with ether1, switch rule was possible and working and got the following :

/interface ethernet switch rule
add dst-port=67 mac-protocol=ip new-vlan-priority=6 ports=switch1-cpu protocol=udp src-mac-address=74:4D:28:6D:F7:AB/FF:FF:FF:FF:FF:FF switch=switch1


What should I do ?
 
nscheffer
newbie
Topic Author
Posts: 45
Joined: Sun Mar 29, 2020 12:52 am

Re: Transparent hEX S to change vlan-priority for DHCP request only

Wed Apr 07, 2021 11:01 pm

I have a problem with mac-protocol=ip in the switch chip rule, because mac-protocol is vlan here if tagging is done before the rule is applied.

In bridge filter rules, the IP and port matching is only possible when mac-protocol=ip, but if I remember correctly, this is not the case with switch rules.
How to detect a switch rule is applied ?
How to debug it ?
 
nscheffer
newbie
Topic Author
Posts: 45
Joined: Sun Mar 29, 2020 12:52 am

Re: Transparent hEX S to change vlan-priority for DHCP request only

Wed Apr 07, 2021 11:12 pm

I have a problem with mac-protocol=ip in the switch chip rule, because mac-protocol is vlan here if tagging is done before the rule is applied.

In bridge filter rules, the IP and port matching is only possible when mac-protocol=ip, but if I remember correctly, this is not the case with switch rules.
on the ether3 the DHCPv4 Request arriving from the Firewall is already tagged with vlanid=832 I just need to change the VLANs-prioirty to 6 that's all before to go trough sfp1 into the Nokia SFP ONT.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Transparent hEX S to change vlan-priority for DHCP request only

Thu Apr 08, 2021 8:50 pm

OK, so I have debugged it locally, setting a switch chip rule to handle tagged traffic for the Tik's own IP address, which therefore that lands at the bridge interface:

dst-address=192.168.6.2/32 dst-port=53 mac-protocol=ip new-vlan-priority=3 ports=ether1 protocol=udp switch=switch1 vlan-id=6

If VLAN tag is present, mac-protocol looks at the ethertype field in the VLAN tag, so my assumption above was wrong. If I change mac-protocol to vlan, the rule stops matching; if mac-protocol is set to ip or if it is not specified at all, the rule matches. If I change the vlan-id from the correct one to something else, the rule stops matching again. It also matches on dst-port=53 even if no mac-protocol is specified. And dst-address may be omitted as well, it still matches on dst-port.

So I can't see what the problem is in your case. Switch chip rules don't provide any statistics, so the only way to debug is to connect something to the egress port externally; you can use an Ethernet interface which is not a member port of any bridge and connect it to an unused member port of the bridge, and sniff on it (I suppose you don't have an SFP with RJ-45 handy).

So e.g. permit vlan 832 on ether1 in addition to ether3 and sfp1 in /interface ethernet switch vlan, remove ether2 from /interface bridge port list and don't attach any IP configuration to it, connect ether1 with ether2 using an external patch cable, make the command line window as wide as your screen permits, and run /tool sniffer quick interface=ether2 in it. Then, connect the firewall to ether3 and see what's going on. In the VLAN column, you should see X or X:Y where X is the VLAN ID and Y is the CoS value if it differs from 0.

The mirroring of sfp1 traffic to ether5 as such works too (except that its slows down forwarding to/from sfp1 as both directions of a 1 Gbit/s link at sfp1 must fit into a single one at ether5, and there are buffers), but the problem is that if you sniff on ether5 using Wireshark running on Windows, you won't see the VLAN tag at all as most Windows network card drivers strip the VLAN tag before handing the frame over to higher levels of the networking stack, including the point where npcap (or winpcap) is hooked. So both the VLAN ID and priority fields are lost. That's why sniffing on Mikrotik itself is necessary, unless you have a modified driver or a linux machine.
 
nscheffer
newbie
Topic Author
Posts: 45
Joined: Sun Mar 29, 2020 12:52 am

Re: Transparent hEX S to change vlan-priority for DHCP request only

Fri Apr 09, 2021 11:38 pm

Hi Sindy,

Sorry for the late reply, I was busy on some works.
I will try to reach a working solution this weekend.
I let you know the result.

Regards,

Nicolas
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Transparent hEX S to change vlan-priority for DHCP request only

Sat Apr 10, 2021 11:31 am

I've realized that my suggestion above regarding sniffing may be overly complex and inconsistent.

To see that the switch chip rule works for the initial DHCPDISCOVER, which is sent to a broadcast address, it is enough to make the bridge a member port of the bridge (see this for clarification of this apparent nonsense) and sniff on the bridge, no need to use an external cable.

To see that the switch chip rule works also for the subsequent DHCPREQUEST, which the client sends to the unicast address of the DHCP server it has chosen among those that have sent it a DHCPOFFER, you either need the port mirroring if connecting to the real uplink via sfp1, or instead you can attach a test DHCP server to a VLAN interface attached to the bridge to see the full process of the firewall obtaining a DHCP lease from this test server.

So to sniff the firewall's dialog with the ISP's DHCP server without losing the VLAN tag in the process, you'd modify the suggestion above: you wouldn't add ether1 to the bridge, and you would connect the mirror-target port (ether5) with the sniffer port (ether2) using an external cable.
 
nscheffer
newbie
Topic Author
Posts: 45
Joined: Sun Mar 29, 2020 12:52 am

Re: Transparent hEX S to change vlan-priority for DHCP request only

Tue May 11, 2021 5:07 pm

Hi Sindy,

Unfortunatly I was not successful to have my internet access working.
I decide to try with OpenWRT giving me more flexibility and features.
Thanks a lot for your help.
Regards,
Nicolas
 
nscheffer
newbie
Topic Author
Posts: 45
Joined: Sun Mar 29, 2020 12:52 am

Re: Transparent hEX S to change vlan-priority for DHCP request only

Wed May 19, 2021 6:04 pm

Hi Sindy,

Unfortunatly I was not successful to have my internet access working.
I decide to try with OpenWRT giving me more flexibility and features.
Thanks a lot for your help.
Regards,
Nicolas
Hi Sindy,
Finally i could be back !!
SFP ONT Nokia G-010S-A are not correctly recognized with the RB2011xxx and was not working with my current Alcatel OLT from my Orange Provider.
It seems SFP ONT Alcatel G-010S-P are working better and could also works into a CRS305 !!!

I would like to do the following architecture :

Fiber 1 Orange -> SFP ONT G-010S-P -> CRS305 SFP+ 1
CRS305 SFP+ 2 -> SFP+ Fortigate Firewall Port 1 SFP+
Fiber 2 Orange -> SFP ONT G-010S-P -> CRS305 SFP+ 3
CRS305 SFP+ 4 -> SFP+ Fortigate Firewall Port 2 SFP+
CRS305 RJ45 PoE In -> Fortigate Firewall Port 3 RJ45 with PoE (used for management)

I suppose I could have the following configuration :
Bridge 1 : Fiber 1 with SFP ONT and Port 1 Fortigate using vlan 832
Bridge 2 : Fiber 2 with SFP ONT and Port 2 Fortigate using vlan 832 (same vlan number but different data)

First could I do this with a CRS305 using 2 Bridge with inside each Bridge a vlan ID 832 ?

Second could I apply a filter to just change vlan Cos to 6 on DHCPv4 Request only ?
The filter is the only processing to be done on the CRS305 all the processing is done on the Fortigate Firewall.
May I keep 1Gbps of traffic per Fiber end to end ?

Thanks in advance.

Reagrds

Nicolas
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Transparent hEX S to change vlan-priority for DHCP request only

Wed May 19, 2021 7:32 pm

Yes, it will work with hardware forwarding, hence at fiber speed. You probably have to make sure no traffic will leak between the two uplinks, or at least ensuring that should cause no harm.

So assuming the management interface of the CRS305 is ether1, the management IP subnet of the CRS305 is attached to bridge, and no VLAN tagging is used at Fortinet side for the management interface, the setup will be as follows:
/interface bridge vlan
add vlan-ids=832 bridge=bridge tagged=sfp-sfpplus1,sfp-sfpplus2,sfp-sfpplus3,sfp-sfpplus4

/interface bridge set bridge vlan-filtering=yes

/interface ethernet switch chip rule
add switch=switch1 ports=sfp-sfpplus2 vlan-id=832 protocol=udp dst-port=67 new-dst-ports=sfp-sfpplus1 new-vlan-priority=6 
add switch=switch1 ports=sfp-sfpplus2 vlan-id=832 new-dst-ports=sfp-sfpplus1

add switch=switch1 ports=sfp-sfpplus1 vlan-id=832 new-dst-ports=sfp-sfpplus2

add switch=switch1 ports=sfp-sfpplus4 vlan-id=832 protocol=udp dst-port=67 new-dst-ports=sfp-sfpplus3 new-vlan-priority=6 
add switch=switch1 ports=sfp-sfpplus4 vlan-id=832 new-dst-ports=sfp-sfpplus3

add switch=switch1 ports=sfp-sfpplus3 vlan-id=832 new-dst-ports=sfp-sfpplus4
I assume when you mention "DHCP request only", it actually means anything the DHCP client sends. Because in the DHCP vernacular, DHCPDISCOVER and DHCPREQUEST are two different messages. And there's no way to distinguish between these two messages by the fields the switch chip rules are able to match on.
 
nscheffer
newbie
Topic Author
Posts: 45
Joined: Sun Mar 29, 2020 12:52 am

Re: Transparent hEX S to change vlan-priority for DHCP request only

Wed May 19, 2021 7:59 pm

One important point each Fiber must be isolated together because they use the same vlan id 832, If I put everyone into the same vlan it will be confusing for all !
So can I create 2 bridge, one for each fiber (SFP ONT and Firewall port) where inside each bridge vlan id 832 is used by the provider to carry internet trafic ?

If it's not possible I coud isolate each Fiber with 2x CRS305 !!

Regarding DHCP "vernacular", yes only the DHCPv4 Request need to be tag Cos 6 to have an answer and get internet up, renew is needed too.
Same for IPv6 where DHCPv6-PD Solicit need to be tag CoS 6.

For management port I was thinking to have it outside the 2 bridge.
 
nscheffer
newbie
Topic Author
Posts: 45
Joined: Sun Mar 29, 2020 12:52 am

Re: Transparent hEX S to change vlan-priority for DHCP request only

Wed May 19, 2021 8:22 pm

If I create two different bridge one with sfpplus1 and sfpplus2, the other sfpplus3 and sfpplus4.
Inside each bridge I add a vlan id 832, same for each bridge.
Does each bridge could act and handle each vlan 832 as separate vlan ?
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Transparent hEX S to change vlan-priority for DHCP request only

Wed May 19, 2021 9:35 pm

Does each bridge could act and handle each vlan 832 as separate vlan ?
Yes, but only one of these bridges can (at least to date) outsource its job to the switch chip, so the other one would forward in software. That's why I've suggested the described solution with port isolation using switch chip rules. These rules do not slow down the processing and allow the frames to be forwarded only between the necessary pairs of ports.
 
nscheffer
newbie
Topic Author
Posts: 45
Joined: Sun Mar 29, 2020 12:52 am

Re: Transparent hEX S to change vlan-priority for DHCP request only

Wed May 19, 2021 10:02 pm

I didn't understood correctly, sorry !
It's more clear and make sense, very good.

Regarding the rule to change Cos 6 to DHCPv4 Request, I suppose I could just add after another one for DHCPv6-PD Solicit ?
add switch=switch1 ports=sfp-sfpplus2 vlan-id=832 protocol=udp dst-port=67 new-dst-ports=sfp-sfpplus1 new-vlan-priority=6
add switch=switch1 ports=sfp-sfpplus2 vlan-id=832 protocol=udp dst-port=547 new-dst-ports=sfp-sfpplus1 new-vlan-priority=6

May I need to specify mac-protocol=ipv6 for the second ?
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Transparent hEX S to change vlan-priority for DHCP request only

Wed May 19, 2021 11:08 pm

I have no idea how the switch chip handles ports with IPv6. I'd assume it is either clever enough to check the ethertype value in the 802.1Q tag and look for port values at appropriate positions in the frame autonomously, or adding mac-protocol=ipv6 won't help either. But if it does adjust to IP type, and you want to make the CoS-setting rules act on port 67 only for IPv4 (and not for IPv6) and, vice versa, act on port 547 only for IPv6, try setting mac-protocol=ip and mac-protocol=ipv6 in both rules respectively.
 
nscheffer
newbie
Topic Author
Posts: 45
Joined: Sun Mar 29, 2020 12:52 am

Re: Transparent hEX S to change vlan-priority for DHCP request only

Wed May 19, 2021 11:29 pm

Ok I receive from Amazon a new CRS305 tomorrow, I try first on IPv4 and when it works try IPv6.
I let you know results.
Thanks a lot again.
Nicolas
 
nscheffer
newbie
Topic Author
Posts: 45
Joined: Sun Mar 29, 2020 12:52 am

Re: Transparent hEX S to change vlan-priority for DHCP request only

Sun Apr 24, 2022 2:29 pm

Yes, it will work with hardware forwarding, hence at fiber speed. You probably have to make sure no traffic will leak between the two uplinks, or at least ensuring that should cause no harm.

So assuming the management interface of the CRS305 is ether1, the management IP subnet of the CRS305 is attached to bridge, and no VLAN tagging is used at Fortinet side for the management interface, the setup will be as follows:
/interface bridge vlan
add vlan-ids=832 bridge=bridge tagged=sfp-sfpplus1,sfp-sfpplus2,sfp-sfpplus3,sfp-sfpplus4

/interface bridge set bridge vlan-filtering=yes

/interface ethernet switch chip rule
add switch=switch1 ports=sfp-sfpplus2 vlan-id=832 protocol=udp dst-port=67 new-dst-ports=sfp-sfpplus1 new-vlan-priority=6 
add switch=switch1 ports=sfp-sfpplus2 vlan-id=832 new-dst-ports=sfp-sfpplus1

add switch=switch1 ports=sfp-sfpplus1 vlan-id=832 new-dst-ports=sfp-sfpplus2

add switch=switch1 ports=sfp-sfpplus4 vlan-id=832 protocol=udp dst-port=67 new-dst-ports=sfp-sfpplus3 new-vlan-priority=6 
add switch=switch1 ports=sfp-sfpplus4 vlan-id=832 new-dst-ports=sfp-sfpplus3

add switch=switch1 ports=sfp-sfpplus3 vlan-id=832 new-dst-ports=sfp-sfpplus4
I assume when you mention "DHCP request only", it actually means anything the DHCP client sends. Because in the DHCP vernacular, DHCPDISCOVER and DHCPREQUEST are two different messages. And there's no way to distinguish between these two messages by the fields the switch chip rules are able to match on.
Hi Sindy,

Finally I bough again a CRS305 with ONU from fs.com to try again to replace my two Orange box and also to avoid double NAT !
On the CRS305 side, i am going to try with one Fiber first with the excellent advice you did above last year (thanks again, it was great !)
I would like to avoid to send the DHCPv4 request with the vlan 832 from the Firewall and would like to catch with a switch rule all ingress traffic on the SFP+ connected with the firewall without the vlan-id=832 and to apply a new-dst-ports=sfp-onu with new-lan-priority=6 and insert the vlan tag id=832, how to do the vlan tag id insertion in the same time with the vlan-priority ?

Many thanks in advance.
Regards
Nicolas
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Transparent hEX S to change vlan-priority for DHCP request only

Sun Apr 24, 2022 4:34 pm

I'm not sure I understand well what you want to do. If the traffic should be sent and received tagged with VID 832 at the ONU SFP port, and received and sent tagless at the Fortigate-facing port, that's a normal behavior, you just configure the Fortigate-facing port as an access one to VLAN 832, and keep VLAN 832 at the SFP port in trunk mode.

If you eventually wanted to assign VID 832 only to DHCP packets but not to other ones coming via the same port, this is not possible with CRS3xx. Maybe CRS1xx can do that, not sure though.

Also I'm a little bit confused regarding how many CRS305 you've got now. In theory one should be sufficient if you have two ports at Fortigate side, as you can use switch chip rules to say that traffic can only be forwarded from SFPa SFPx, from SFPx to SFPa, from SFPb to SFPy, and from SFPy to SFPb. So that way everything works in hardware but still the two uplinks are isolated.

With a single port on the Fortigate, it might be doable using VLAN ID translation, where the Fortigate would use two other VLANs than 832, one per each uplink, and you would use VLAN ID translating rules at ingress - what comes from the Fortigate in any of these additional VLANs would be translated to 832, and whetever comes from the ISPs in 832 would be translated to one of these additional VLANs. And of course the forwarding permissions would have to be SFPfg -> (SFPisp1, SFPisp2), SFPisp1->SFPfg, SFPIsp2->SFPfg.

So the DHCPDISCOVER from either auxiliary VLAN would reach both ISPs, but the response from the "wrong" one would end up in a "wrong" VLAN at FG side and get ignored (even if it came at the "right" time while a response to DHCPDISOVER was awaited, it would still bear a wrong transaction ID).
 
nscheffer
newbie
Topic Author
Posts: 45
Joined: Sun Mar 29, 2020 12:52 am

Re: Transparent hEX S to change vlan-priority for DHCP request only

Sun Apr 24, 2022 7:25 pm

Hi Sindy,

Thank you for your very quick answer, Mikrotik support is rock !
All my current Mikrotik devices are used, it's why I bough a new CRS305 for this come back...

Actually on the Firewall, I got each Livebox connected to a physical port and each wan is mapped into a separate vlan (11 and 12).
If I replace one Livebox with built-in ONU I need to add the ONU from fs.com into the CRS305 into one SFP+ port and took another SFP+ port with a S+RJ10 on the CRS305 to go on the Firewall.
But I would like to have the vlan 832 required on the ONU side with the DHCPv4 request tagged with CoS 6 translated to the existing vlan on the firewall side (11 or 12), that's my request.
Only DHCPv4 request and renew (all DHCPv4 traffic in fact) needs to be set with CoS 6 into vlan 832 and all other traffic need to be into vlan 832 too.

If I am correct and following your answer we can do it in 2 different way :
- vlan 832 traffic inside the ONU side mapped to the whole port on the firewall side (untagged)
or
- vlan 832 traffic inside the ONU side translated to the current vlan (11 or 12) on the firewall side

Both could be done or not ?
Yes you point a good optimization using one link only with the Firewall from the CRS305 and carry inside both wan (vlan 11 and 12 on the firewall side but vlan 832 for both wan with both ONU)

To be sure, inside the same switch rule ingress traffic on the firewall port I can detect a specific vlan (like 11 or 12), apply CoS 6 to DHCP trafic and translate vlan id to 832 and forward trafic for vlan 11 to port 2 and vlan 12 to port 3, all other non DHCP traffic translate vlan id to 832 and forward trafic for vlan 11 to port 2 and vlan 12 to port 3, is it correct ?
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Transparent hEX S to change vlan-priority for DHCP request only

Sun Apr 24, 2022 7:58 pm

Mikrotik support is rock !
A forum like this is the second best thing to an actual support for the users, but if Mikrotik was to provide this amount of support directly, you would have to pay a lot for a support contract or the equipment would have to cost much more (look at Cisco, there's both, the equipment is expensive and the access to service desk is paid as well).

Both could be done or not ?
Yes, both.

Yes you point a good optimization using one link only with the Firewall from the CRS305 and carry inside both wan (vlan 11 and 12 on the firewall side but vlan 832 for both wan with both ONU)
But doing so limits the total uplink bandwidth to the one of a single port.

To be sure, inside the same switch rule ingress traffic on the firewall port I can detect a specific vlan (like 11 or 12), apply CoS 6 to DHCP trafic and translate vlan id to 832 and forward trafic for vlan 11 to port 2 and vlan 12 to port 3, all other non DHCP traffic translate vlan id to 832 and forward trafic for vlan 11 to port 2 and vlan 12 to port 3, is it correct ?
Correct. Switch rules can do multiple "actions" simultaneously - change CoS, change VID, and restrict the list of permitted egress ports.
 
nscheffer
newbie
Topic Author
Posts: 45
Joined: Sun Mar 29, 2020 12:52 am

Re: Transparent hEX S to change vlan-priority for DHCP request only

Sun Apr 24, 2022 8:13 pm

Perfect, I am all good !
For the limitation with a single link at 10G it's not really a problem because I am using SFP ONU limited to 2.5G max each, so dual will be maximum at 5G so enough bandwidth.
CRS305 has been ordered and I should have it in less than one week, but for the ONU from fs.com actually they told me there is a delay 16 to 20 days...
I let you know as soon as I got everything to start.
Thanks again.
 
nscheffer
newbie
Topic Author
Posts: 45
Joined: Sun Mar 29, 2020 12:52 am

Re: Transparent hEX S to change vlan-priority for DHCP request only

Mon May 16, 2022 7:41 pm

Hi Sindy,

Finally ONU GPON from fs.com will be delivered this Friday, keep you updated with final configuration starting this weekend...
Tuning the config, should I enable the ingress filtering on all sfpplusx ports ?

Here is my current config :
/interface bridge
add admin-mac=18:FD:74:00:44:70 auto-mac=no comment="Internet ONUs" ingress-filtering=no name=bridge vlan-filtering=yes
add comment="Mgt FortiSwitch PoE" name=mgt
/interface ethernet
set [ find default-name=ether1 ] comment=FortiSwitch
set [ find default-name=sfp-sfpplus1 ] comment=FortiGate
set [ find default-name=sfp-sfpplus2 ] comment="Not used"
set [ find default-name=sfp-sfpplus3 ] comment=Orange
set [ find default-name=sfp-sfpplus4 ] comment=Orange-Pro
/interface bridge port
add bridge=bridge comment=defconf ingress-filtering=no interface=sfp-sfpplus1
add bridge=bridge comment=defconf ingress-filtering=no interface=sfp-sfpplus2
add bridge=bridge comment=defconf ingress-filtering=no interface=sfp-sfpplus3
add bridge=bridge comment=defconf ingress-filtering=no interface=sfp-sfpplus4
add bridge=mgt ingress-filtering=no interface=ether1
/interface bridge vlan
add bridge=bridge tagged=sfp-sfpplus1 vlan-ids=111
add bridge=bridge tagged=sfp-sfpplus1 vlan-ids=112
add bridge=bridge tagged=sfp-sfpplus3,sfp-sfpplus4 vlan-ids=832
/interface ethernet switch rule
add comment="Intercept DHCPv4 Request Orange Vlan 111 port sfp1 and forward with vlan 832 and CoS 6 to port sfp3" dst-port=67 new-dst-ports=sfp-sfpplus3 new-vlan-id=832 new-vlan-priority=6 ports=sfp-sfpplus1 protocol=udp switch=switch1 vlan-id=111
add comment="Intercept All Trafic Orange Vlan 111 port sfp1 and forward with vlan 832 and CoS 6 to port sfp3" new-dst-ports=sfp-sfpplus3 new-vlan-id=832 ports=sfp-sfpplus1 switch=switch1 vlan-id=111
add comment="Intercept DHCPv4 Request Orange Pro Vlan 112 port sfp1 and forward with vlan 832 and CoS 6 to port sfp4" dst-port=67 new-dst-ports=sfp-sfpplus4 new-vlan-id=832 new-vlan-priority=6 ports=sfp-sfpplus1 protocol=udp switch=switch1 vlan-id=112
add comment="Intercept All Trafic Orange Pro Vlan 112 port sfp1 and forward with vlan 832 and CoS 6 to port sfp4" new-dst-ports=sfp-sfpplus4 new-vlan-id=832 ports=sfp-sfpplus1 switch=switch1 vlan-id=112
add comment="Intercept All Trafic back from Orange Vlan 832 port sfp3 and forward with vlan 111 to port sfp1" new-dst-ports=sfp-sfpplus1 new-vlan-id=111 ports=sfp-sfpplus3 switch=switch1 vlan-id=832
add comment="Intercept All Trafic back from Orange Pro Vlan 832 port sfp4 and forward with vlan 112 to port sfp1" new-dst-ports=sfp-sfpplus1 new-vlan-id=112 ports=sfp-sfpplus4 switch=switch1 vlan-id=832
/ip dhcp-client
add interface=mgt
/ip service
set telnet disabled=yes
set ftp disabled=yes
set www disabled=yes
set ssh port=1122
set www-ssl certificate=crs-chanet disabled=no port=11443
set api disabled=yes
set winbox disabled=yes
set api-ssl disabled=yes
/system package update
set channel=testing
/system routerboard settings
set boot-os=router-os
/tool sniffer
set filter-interface=sfp-sfpplus1 streaming-enabled=yes streaming-server=10.255.16.222
I am using the 7.3 Beta 40 to have the 2.5Gbps ONU GPON link in the sfpplus port !
 
nscheffer
newbie
Topic Author
Posts: 45
Joined: Sun Mar 29, 2020 12:52 am

Re: Transparent hEX S to change vlan-priority for DHCP request only

Wed May 18, 2022 7:50 pm

Hi Sindy,

I don't know if it's the latest beta 7.3 Beta 40 but I got today (2 days in advance !) GPON ONU from fs.com.
I try just one and it seems Switch Rule are not working anymore !

I check with my laptop using Wireshark on the FortiGate 10Gbs interface and I got with respective vlan 111 and 112 DHCP Request for Orange and Orange-Pro.
Sfpplus3 port on the CRS305 I should have the DHCPv4 Request coming from vlan 111 on port sfpplus1 with vlan changed to 832 with vlan-priority set to 6 but I got nothing !
How to troubleshoot Switch Rule ?
 
nscheffer
newbie
Topic Author
Posts: 45
Joined: Sun Mar 29, 2020 12:52 am

Re: Transparent hEX S to change vlan-priority for DHCP request only

Wed May 18, 2022 11:01 pm

Hi Sindy,

Finally I solved it, GPON ONU serial was incorrect !!!
Switch Rule diagnostic is not easy, no stats, no logs, nothing, ocular due improved in the future !

I am going to create a full documentation about all my setup, share it internally at Fortinet and post the config on the French forum lafibre.info

Thanks a lot.
Here is my config used for both Fiber (Orange and Orange Pro) :
# may/18/2022 21:51:27 by RouterOS 7.3beta40
# software id = 3WGM-VXU2
#
# model = CRS305-1G-4S+
# serial number = HCB08F5PJ0G
/interface bridge
add admin-mac=18:FD:74:00:44:70 auto-mac=no comment="Internet ONUs" ingress-filtering=no name=bridge vlan-filtering=yes
add comment="Mgt FortiSwitch PoE" name=mgt
/interface ethernet
set [ find default-name=ether1 ] comment=FortiSwitch
set [ find default-name=sfp-sfpplus1 ] comment="FortiGate Orange & Orange-Pro"
set [ find default-name=sfp-sfpplus2 ] auto-negotiation=no comment="ONU Orange" speed=2.5Gbps
set [ find default-name=sfp-sfpplus3 ] comment="Not Used"
set [ find default-name=sfp-sfpplus4 ] auto-negotiation=no comment="ONU Orange-Pro" speed=2.5Gbps
/interface lte apn
set [ find default=yes ] ip-type=ipv4 use-network-apn=no
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/port
set 0 name=serial0
/interface bridge port
add bridge=bridge comment=defconf ingress-filtering=no interface=sfp-sfpplus1
add bridge=bridge comment=defconf ingress-filtering=no interface=sfp-sfpplus2
add bridge=bridge comment=defconf ingress-filtering=no interface=sfp-sfpplus3
add bridge=bridge comment=defconf ingress-filtering=no interface=sfp-sfpplus4
add bridge=mgt ingress-filtering=no interface=ether1
/ip settings
set max-neighbor-entries=8192
/interface bridge vlan
add bridge=bridge comment="Orange & Orange-Pro ONUs" tagged=sfp-sfpplus2,sfp-sfpplus4 vlan-ids=832
add bridge=bridge comment="FortiGate Orange" tagged=sfp-sfpplus1 vlan-ids=111
add bridge=bridge comment="FortiGate Orange-Pro" tagged=sfp-sfpplus1 vlan-ids=112
/interface ethernet switch rule
add comment="Intercept Orange DHCPv4 Request with vlan 111 on port sfp1 and forward it with vlan 832 and CoS 6 to port sfp2" dst-port=67 new-dst-ports=sfp-sfpplus2 new-vlan-id=832 new-vlan-priority=6 ports=sfp-sfpplus1 protocol=udp switch=switch1 vlan-id=111
add comment="Intercept All Orange Trafic with vlan 111 on port sfp1 and forward it with vlan 832 and CoS 6 to port sfp2" new-dst-ports=sfp-sfpplus2 new-vlan-id=832 ports=sfp-sfpplus1 switch=switch1 vlan-id=111
add comment="Intercept All Orange Trafic back from Vlan 832 on port sfp2 and forward it to port sfp1 with vlan 111" new-dst-ports=sfp-sfpplus1 new-vlan-id=111 ports=sfp-sfpplus2 switch=switch1 vlan-id=832
add comment="Intercept Orange-Pro DHCPv4 Request with vlan 112 on port sfp1 and forward it with vlan 832 and CoS 6 to port sfp4" dst-port=67 new-dst-ports=sfp-sfpplus4 new-vlan-id=832 new-vlan-priority=6 ports=sfp-sfpplus1 protocol=udp switch=switch1 vlan-id=112
add comment="Intercept All Orange-Pro Trafic with vlan 112 on port sfp1 and forward it with vlan 832 and CoS 6 to port sfp4" new-dst-ports=sfp-sfpplus4 new-vlan-id=832 ports=sfp-sfpplus1 switch=switch1 vlan-id=112
add comment="Intercept All Orange-Pro Trafic back from Vlan 832 on port sfp4 and forward it to port sfp1 with vlan 112" new-dst-ports=sfp-sfpplus1 new-vlan-id=112 ports=sfp-sfpplus4 switch=switch1 vlan-id=832
/interface ovpn-server server
set auth=sha1,md5
/ip dhcp-client
add interface=mgt
/ip service
set telnet disabled=yes
set ftp disabled=yes
set www disabled=yes
set ssh port=1122
set www-ssl certificate=crs-chanet disabled=no port=11443
set api disabled=yes
set winbox disabled=yes
set api-ssl disabled=yes
/system clock
set time-zone-name=Europe/Paris
/system identity
set name=CRS305
/system package update
set channel=testing
/system routerboard settings
set boot-os=router-os
/tool sniffer
set filter-interface=sfp-sfpplus1 streaming-enabled=yes streaming-server=10.255.16.222
 
nscheffer
newbie
Topic Author
Posts: 45
Joined: Sun Mar 29, 2020 12:52 am

Re: Transparent hEX S to change vlan-priority for DHCP request only

Thu May 19, 2022 12:00 am

Hi Sindy,

I got one problem !
It seems both Fiber cannot be handle in the same time, Switch Rule could process TX and RX flow for one Fiber but on the second only TX is process not RX back packets...
Flags: R - RUNNING; S - SLAVE
Columns: NAME, RX-BYTE, TX-BYTE, RX-PACKET, TX-PACKET, RX-DROP, TX-DROP, TX-QUEUE-DROP, RX-ERROR, TX-ERROR
#    NAME              RX-BYTE      TX-BYTE  RX-PACKET  TX-PACKET  RX-DROP  TX-DROP  TX-QUEUE-DROP  RX-ERROR  TX-ERROR
;;; FortiSwitch
0 RS ether1            898 096    2 251 346      3 696      2 421                                0                    
;;; FortiGate Orange & Orange-Pro
1 RS sfp-sfpplus1  124 339 651  130 395 486    154 075    185 862                                0                    
;;; ONU Orange
2 RS sfp-sfpplus2  114 841 217  118 956 857    172 224    146 234                                0                    
;;; Not Used
3  S sfp-sfpplus3            0            0          0          0                                0                    
;;; ONU Orange-Pro
4 RS sfp-sfpplus4   15 553 730    5 392 074     13 642      7 997                                0                    
;;; Internet ONUs
5 R  bridge                  0          220          0          2        0        0              0         0         0
;;; Mgt FortiSwitch PoE
6 R  mgt               854 066    2 240 803      3 542      2 411        0        0              0         0         0
ONU Orange (sfpplus2) has both TX/RX traffic process with the Switch Rule appropriated and ONU Orange Pro (sfpplus4) has some TX flow but RX is equal to zero !
Why ?
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Transparent hEX S to change vlan-priority for DHCP request only

Fri May 20, 2022 8:18 am

I'm not sure I get you right - the statistics you've posted shows both sfp-sfpplus2 and sfp-sfpplus2 to handle both Tx and Rx traffic. Is it that you have some other statistics on the Fortinet that says that there is no Rx on VLAN 112, i.e. that the CRS does not forward frames received at sfp-sfpplus4 to sfp-sfpplus1?
Flags: R - RUNNING; S - SLAVE
Columns: NAME, RX-BYTE, TX-BYTE, RX-PACKET, TX-PACKET, RX-DROP, TX-DROP, TX-QUEUE-DROP, RX-ERROR, TX-ERROR
#    NAME              RX-BYTE      TX-BYTE  RX-PACKET  TX-PACKET  RX-DROP  TX-DROP  TX-QUEUE-DROP  RX-ERROR  TX-ERROR
;;; ONU Orange
2 RS sfp-sfpplus2  114 841 217  118 956 857    172 224    146 234                                0                    
;;; ONU Orange-Pro
4 RS sfp-sfpplus4   15 553 730    5 392 074     13 642      7 997                                0                    
In any case, to find out whether it's something about the order of the rules or something about the actual behaviour of the Orange uplinks, you can keep the order of the rules, swap vlans 111 and 112 in all of them, and swap the patchcords. But I cannot see anything to be missing in the rules:
add comment="Intercept Orange DHCPv4 Request with vlan 111 on port sfp1 and forward it with vlan 832 and CoS 6 to port sfp2" \
    switch=switch1 ports=sfp-sfpplus1 vlan-id=111 protocol=udp dst-port=67 new-dst-ports=sfp-sfpplus2 new-vlan-id=832 new-vlan-priority=6
add comment="Intercept All Orange Trafic with vlan 111 on port sfp1 and forward it with vlan 832 and CoS 6 to port sfp2" \
    switch=switch1 ports=sfp-sfpplus1 vlan-id=111                           new-dst-ports=sfp-sfpplus2 new-vlan-id=832
add comment="Intercept All Orange Trafic back from Vlan 832 on port sfp2 and forward it to port sfp1 with vlan 111" \
    switch=switch1 ports=sfp-sfpplus2 vlan-id=832                           new-dst-ports=sfp-sfpplus1 new-vlan-id=111
add comment="Intercept Orange-Pro DHCPv4 Request with vlan 112 on port sfp1 and forward it with vlan 832 and CoS 6 to port sfp4" \
    switch=switch1 ports=sfp-sfpplus1 vlan-id=112 protocol=udp dst-port=67 new-dst-ports=sfp-sfpplus4 new-vlan-id=832 new-vlan-priority=6
add comment="Intercept All Orange-Pro Trafic with vlan 112 on port sfp1 and forward it with vlan 832 and CoS 6 to port sfp4" \
    switch=switch1 ports=sfp-sfpplus1 vlan-id=112                           new-dst-ports=sfp-sfpplus4 new-vlan-id=832
add comment="Intercept All Orange-Pro Trafic back from Vlan 832 on port sfp4 and forward it to port sfp1 with vlan 112" \
    switch=switch1 ports=sfp-sfpplus4 vlan-id=832                           new-dst-ports=sfp-sfpplus1 new-vlan-id=112[code]
 
nscheffer
newbie
Topic Author
Posts: 45
Joined: Sun Mar 29, 2020 12:52 am

Re: Transparent hEX S to change vlan-priority for DHCP request only

Fri May 20, 2022 10:25 am

Hi Sindy,

First thanks for your reply.
Let me try to give you a summary about what I did and learn :

I try 2 different architecture with same result :
1) Single connection to Firewall
- a DAC cable on sfpplus1 going to firewall to carry wan 1 (with vlan 111 for orange tagged) and and wan2 (with vlan 112 for orange-pro tagged)
- ONU GPON for orange on sfpplus2
- ONU GPON for orange-pro on sfpplus4

2) a connection per wan to Firewall
- a DAC cable on sfpplus1 going to firewall to carry wan 1 (with vlan 111 for orange tagged )
- a DAC cable on sfpplus3 going to firewall to carry wan 3 (with vlan 112 for orange-pro tagged )
- ONU GPON for orange on sfpplus2
- ONU GPON for orange-pro on sfpplus4
- updated Switch Rules to handle sfpplus3

Actually I am running with first architecture but on both architecture I have the same behavior, no difference.

What I notice :
- Firewall can connect for each wan because dhcpv4 request are complete don both and I got for each public IP
- only last DHCP request seems to be the working and active wan with Switch Rules

Look at the screenshot ("current with orange-pro") only Orange-Pro is working (CRS305 has RX/TX for wan 2 orange-pro and no RX at all for wan 1 orange, Fortinet Firewall has a status up for wan 2 orange-pro and send traffic to it, wan 1 is detected has down).

Now if I force a DHCP Renew on wan 1 Orange (screenshot "force dhcp renew on orange"), CRS305 has RX flow back for wan 1 but dead for wan2, Fortinet Firewall consider now the wan 1 up and running and the wan 2 dead !!!

It seems the Switch Rule cannot process both wan together in the same time, why ?
Let me know what you need for more details.
You do not have the required permissions to view the files attached to this post.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Transparent hEX S to change vlan-priority for DHCP request only

Fri May 20, 2022 11:08 am

Observation: the behaviour you describe is a stateful one - a single DHCP renewal changes how multiple subsequent packets are handled.
Fact: there is nothing stateful about the switch chip rules themselves.
Conclusion: the behaviour you observe is not caused by switch chip rules.

Assumption: the L2 network of Orange is a common one for both uplinks, and it receives frames from the same MAC address (of the firewall) in the same VLAN via two different uplinks, so it keeps sending frames for that MAC address through the uplink on which it has received a frame from that MAC address most recently. Maybe this is not caused directly by MAC learning but linked to the DHCP lease. If you can make the firewall use a different MAC address for each DHCP client/uplink interface, it should help. At least when using Architecture 2, it should be possible not to bridge the two interfaces of the firewall together internally.
 
nscheffer
newbie
Topic Author
Posts: 45
Joined: Sun Mar 29, 2020 12:52 am

Re: Transparent hEX S to change vlan-priority for DHCP request only

Fri May 20, 2022 11:37 am

Switch Rules is not easy to troubleshoot, no stats no monitoring and If I am correct when there is a Switch Rule matching I cannot have packet capture with streaming ?

Here is a diagram for the first scenario, one cable to share both wan.
I am going to check on the Firewall too if there is nothing wrong.

As you mention there is no commun part for both Orange Fiber Internet access (Orange and Orange Pro), even when they share the same cable between the CRS305 and the Firewall they are each on different vlans.

Your last point regarding Architecture 2, if I don't put all of them into a single bridge Switch Rules will work for ports not into a bridge ?
You do not have the required permissions to view the files attached to this post.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Transparent hEX S to change vlan-priority for DHCP request only

Fri May 20, 2022 12:06 pm

Switch Rules is not easy to troubleshoot, no stats no monitoring and If I am correct when there is a Switch Rule matching I cannot have packet capture with streaming ?
Correct. I think you can sniff if you force copying of the ingress frames to the CPU, but you have to make sure the traffic volume is low in order not to overload the CPU.

As you mention there is no commun part for both Orange Fiber Internet access (Orange and Orange Pro), even when they share the same cable between the CRS305 and the Firewall they are each on different vlans.
What I had in mind was a single common network at Orange side. In the beginning, it wasn't clear that both uplinks were provided by the same ISP, so I did not take these L2 issues into consideration. If you can use Architecture 2 and connect something else than the firewall to sfp-sfpplus1, you should see immediately whether the issue is related to the common MAC address at your end of both uplinks or not.
 
nscheffer
newbie
Topic Author
Posts: 45
Joined: Sun Mar 29, 2020 12:52 am

Re: Transparent hEX S to change vlan-priority for DHCP request only

Fri May 20, 2022 6:00 pm

I am gong to try again Architecture 2 and I will pay attention on the Firewall to see if there is no blocking point !
On the CRS305 can I force a different MAC when I will use sfpplus3 to carry the second wan to the Firewall ?
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Transparent hEX S to change vlan-priority for DHCP request only

Fri May 20, 2022 6:09 pm

On the CRS305 can I force a different MAC when I will use sfpplus3 to carry the second wan to the Firewall ?
No, you cannot - there is no switch chip rule that can change the source MAC address of a frame, let alone that it could change it inside the payload of the DHCP packets where it is probably used as the Client ID field. But if you configure the two WAN ports of the firewall as separate IP interfaces, they should use distinct MAC addresses automatically. Regarding Architecture 1, there may be a possibility to assign a MAC address to the interface in VLAN 112 manually, but I don't know Fortigate enough to tell you whether it is indeed possible and if yes, how exactly to configure it.
 
nscheffer
newbie
Topic Author
Posts: 45
Joined: Sun Mar 29, 2020 12:52 am

Re: Transparent hEX S to change vlan-priority for DHCP request only

Fri May 20, 2022 6:26 pm

Architecture 1 - I am going to try to force a different MAC for each vlan of each wan on the Firewall side and let you know the result.
Architecture 2 - I just try again 5mn again and this time both are blocked, none are working !!!! I suspect the Firewall to see the same MAC on different port and/or stp loop... I investigate too
 
nscheffer
newbie
Topic Author
Posts: 45
Joined: Sun Mar 29, 2020 12:52 am

Re: Transparent hEX S to change vlan-priority for DHCP request only

Fri May 20, 2022 10:43 pm

Hi Sindy,

You are wright, using Architecture 1 it seems the Firewall is blocking one wan port (but I cannot identify it and confirm it, could be the CRS305 also blocking alternatively one port), why I don't know.
I just open a ticket at Fortinet and also at Mikrotik (SUP-82564) providing the link to this thread for info.

Actually the problem is correctly located on Rx flow for sfpplus2 or 4 (with the GPON ONU SFP) and I don't how to explain if it's the CRS305 who connote match and/or apply the Switch Rule to send back traffic to the Fortigate and/or if it's the ForitGate who is blocking for an unknown reason the Rx flow back from GPON ONU SFP trough CRS305 and going to Firewall FortiGate...

I let you know the progress and the result.

Have a great weekend.
Nicolas
 
nscheffer
newbie
Topic Author
Posts: 45
Joined: Sun Mar 29, 2020 12:52 am

Re: Transparent hEX S to change vlan-priority for DHCP request only

Mon May 23, 2022 6:53 pm

Hi Sindy,

I did many check and try to understand what's wrong and I suspect that CRS305 (or Firewall) does not accept packet back from ONU, you see TX packets going to ONU from CRS305 but the switch rule who is supposed to catch traffic back from ONU to Firewall is resulting a zero in term of packet per seconds or Firewall denies packets to come back !!!
This is true for the wan not working, the other wan switch rules are correctly processing packet back to Firewall or Firewall accept packets back...

By the way I open a ticket 4 days, still no answer ?
 
nscheffer
newbie
Topic Author
Posts: 45
Joined: Sun Mar 29, 2020 12:52 am

Re: Transparent hEX S to change vlan-priority for DHCP request only

Tue May 24, 2022 12:08 pm

Hi Sindy,

I did a chat session today with Fortinet support and after many tests and verifications it seems the Mikrotik CRS305 is not able to handle both Fiber with Switch Rules !
Here is what we agree :
- i am gooing to receive a second CRS305 (I order it yesterday) to do the following
- scenario 1, I will use the second Mikrotik to insert it as a passive switch to snif trafic between the GPON ONU SFP and the CRS305 where it was inserted, I will do the same to snif trafic between CRS305 and the Fortinet Switch.
- scenario 2, I will use the second Mikrotik to handle the second Fiber, one CRS305 per Fiber !!!!

I let you know about the result.

Regards,
Nicolas
 
nscheffer
newbie
Topic Author
Posts: 45
Joined: Sun Mar 29, 2020 12:52 am

Re: Transparent hEX S to change vlan-priority for DHCP request only

Sun May 29, 2022 12:28 pm

Hi Sindy,

Quick update, I restart from scratch all my config and in the same time got feedback on my case with Mikrotik support.
It seems the problem was with MAC learning enabled by default on each port inside the bridge, by disabling each I got now one Fiber with the ONU GPON into the CRS305 connected to the Firewall working perfectly and the other Fiber using the original box connected directly to the firewall.
I am going to wait a few days like this and if everything is ok I will move the second Fiber trough the CRS with the ONU GPON...
Keep you updated.
 
nscheffer
newbie
Topic Author
Posts: 45
Joined: Sun Mar 29, 2020 12:52 am

Re: Transparent hEX S to change vlan-priority for DHCP request only

Wed Jun 15, 2022 2:47 pm

Hi Sindy,

Not enough time and competences to correctly investigate and troubleshoot the problem when using both Fiber without their Telcos Box using GPON ONU directly on CRS305 SFP+ ports.
Problem was the same with one CRS305 handling both Fiber and two CRS305 for each Fiber, the Firewall Fortinet (FortiGate 101F running 7.2) was doing the same behavior blocking one fiber vs the other running and If you force a DCHP renew on the blocked Fiber we got a switch of the running Fiber were blocked and the blocked Fiber went running...
Very strange I don't know why, ideas I got in mind :
- same OLT for both Fiber maybe giving trouble for the Firewall Fortinet ?
- bugs on the Firewall ? I open a ticket and first investigation seems not !
- bugs on CRS ? Why we have it with one or two CRS305 the same problem ?
- other ideas ?
Last edited by nscheffer on Wed Jun 15, 2022 2:54 pm, edited 1 time in total.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Transparent hEX S to change vlan-priority for DHCP request only

Wed Jun 15, 2022 2:54 pm

With two CRS in the path, what MAC addresses does the Fortinet use on the two uplinks, same or different? As I've sugested before, frames from the same MAC address arriving to the ISP switch via two distinct physical links may confuse it.
 
nscheffer
newbie
Topic Author
Posts: 45
Joined: Sun Mar 29, 2020 12:52 am

Re: Transparent hEX S to change vlan-priority for DHCP request only

Wed Jun 15, 2022 2:58 pm

Didn't check and/or pay attention when I did it.
I can use EMAC VLAN (Enhanced MAC) on the Firewall for each Fiber to force different MAC but I am loosing 10Gbps speed (don't have engohu SFP+ port on the Firewall), but I can aggregate two Gig port if success without loosing the full 2Gbps speed on each Fiber....
Let you know.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Transparent hEX S to change vlan-priority for DHCP request only

Wed Jun 15, 2022 3:04 pm

Can EMAC VLAN be used on a single 10 GB fiber, i.e. each VLAN interface on the same physical port would have a different MAC?
 
nscheffer
newbie
Topic Author
Posts: 45
Joined: Sun Mar 29, 2020 12:52 am

Re: Transparent hEX S to change vlan-priority for DHCP request only

Wed Jun 15, 2022 3:06 pm

Yes this is the goal where each Fiber has a dedicated EMAC and CRS305 (one or two) swap original VLAN on Fortigate side (11 or 12) to the same number (832) for each Fiber adding also CoS 6 on DHCPv4 request...
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Transparent hEX S to change vlan-priority for DHCP request only

Wed Jun 15, 2022 3:10 pm

So if you can set an arbitrary MAC address, choose one of some device you own and you know you will never connect it directly to the Orange fiber, such as the one of your phone's WiFi interface, as if you used some randomly chosen one, you could cause the same issue with some other Orange user.
 
nscheffer
newbie
Topic Author
Posts: 45
Joined: Sun Mar 29, 2020 12:52 am

Re: Transparent hEX S to change vlan-priority for DHCP request only

Sat Jun 18, 2022 7:48 am

Hi Sindy,

Finally it's more complex than I was expecting, missing SFP+ port on FortiGate to enable EMAC, EMAC is not possible at the Switch I got where I got plenty of SFP+ ports !
Need to re-organize my setup....

Who is online

Users browsing this forum: arm920t, ccrsxx and 50 guests