On MT7621 based routers, RouterOS doesn't support switch chip rules at all, not only on sfp1.Agree that the hEX S will be the bottleneck and switch rules are not supported for vlan and also for sfp1.
Both should work fine at wire speed if you use switch chip rules.I plan to replace my hEX S for each box by :
- RB2011iLS-IN
- or RB935GS-5HnT-RP
According to the product information, CRS305-1G-4S+IN seems to be the best choice for your special purpose - one SFP+ towards the firewall, plain SFP towards the slower WAN, and another SFP+ towards the 2G/600M WAN if the firewall supports VLANs. But I've got no hands-on experience with this device. Don't forget to disable xSTP before connecting it to the ISPs.I got 2 fiber access and would like to replace both box, one is 2Gbps/600Mbps and the other 1Gbps/1Gbps.
...
What is your advise for best Mikrotik product to fit this needs as a transparent sfp/ether CoS tagging device ?
What is your advise is I would like to have a sfp+ port instead ? (SFP ONT used is a Nokia G-010S-A with HSGMII at 2,5Gbps to be able to use the 2Gbps available on the fiber contract)
So much for standards sigh. :-(Mikrotik support for ONT SFPs is non existent so some might work and most don't. Even compatibility with "normal" SFPs is incomplete (mildly put). Which means that trying to get ONT SFP to work with any MT device is similar to trying to win a jackpot, even if particular ONT SFP works with one model of MT, it might not with another model of MT.
Sindy,On MT7621 based routers, RouterOS doesn't support switch chip rules at all, not only on sfp1.Agree that the hEX S will be the bottleneck and switch rules are not supported for vlan and also for sfp1.
Both should work fine at wire speed if you use switch chip rules.I plan to replace my hEX S for each box by :
- RB2011iLS-IN
- or RB935GS-5HnT-RP
Hi Sindy,As you want it wirespeed, the switch chip must do all the job. Hence you have to make sfp1 and etherX member ports of a bridge with hw=yes on the respective /interface bridge port rows, and if you ever add any other port of that switch chip to any other bridge, it must be added with hw=no (only a single bridge can be "hardware accelerated" per switch chip, and RouterOS chooses autonomously if you permit hw=yes on multiple bridges).
If I remember it properly, you don't need to configure VLAN 832 handling in the bridge configuration, just set
/interface ethernet switch port set etherX vlan-mode=secure default-vlan-id=832
/interface ethernet switch port set sfp1 vlan-mode=secure
/interface ethernet switch vlan add ports=etherX,sfp1 switch=switch1 vlan-id=832
/interface ethernet switch rule add dst-port=67 new-vlan-priority=6 ports=etherX protocol=udp switch=switch1 vlan-id=832
In the unlikely case that the VLAN tag is added on ingress after the frame has been processed by the rules (I'm not 100 % sure here and can't test right now), remove the vlan-id=832 from the match conditions of the rule. But in your case matching on vlan-id doesn't matter much as no other VLANs will be used on the switch chip anyway.
Hi Sindy,As you want it wirespeed, the switch chip must do all the job. Hence you have to make sfp1 and etherX member ports of a bridge with hw=yes on the respective /interface bridge port rows, and if you ever add any other port of that switch chip to any other bridge, it must be added with hw=no (only a single bridge can be "hardware accelerated" per switch chip, and RouterOS chooses autonomously if you permit hw=yes on multiple bridges).
If I remember it properly, you don't need to configure VLAN 832 handling in the bridge configuration, just set
/interface ethernet switch port set etherX vlan-mode=secure default-vlan-id=832
/interface ethernet switch port set sfp1 vlan-mode=secure
/interface ethernet switch vlan add ports=etherX,sfp1 switch=switch1 vlan-id=832
/interface ethernet switch rule add dst-port=67 new-vlan-priority=6 ports=etherX protocol=udp switch=switch1 vlan-id=832
In the unlikely case that the VLAN tag is added on ingress after the frame has been processed by the rules (I'm not 100 % sure here and can't test right now), remove the vlan-id=832 from the match conditions of the rule. But in your case matching on vlan-id doesn't matter much as no other VLANs will be used on the switch chip anyway.
Hi Sindy,I have a problem with mac-protocol=ip in the switch chip rule, because mac-protocol is vlan here if tagging is done before the rule is applied.
In bridge filter rules, the IP and port matching is only possible when mac-protocol=ip, but if I remember correctly, this is not the case with switch rules.
How to detect a switch rule is applied ?I have a problem with mac-protocol=ip in the switch chip rule, because mac-protocol is vlan here if tagging is done before the rule is applied.
In bridge filter rules, the IP and port matching is only possible when mac-protocol=ip, but if I remember correctly, this is not the case with switch rules.
on the ether3 the DHCPv4 Request arriving from the Firewall is already tagged with vlanid=832 I just need to change the VLANs-prioirty to 6 that's all before to go trough sfp1 into the Nokia SFP ONT.I have a problem with mac-protocol=ip in the switch chip rule, because mac-protocol is vlan here if tagging is done before the rule is applied.
In bridge filter rules, the IP and port matching is only possible when mac-protocol=ip, but if I remember correctly, this is not the case with switch rules.
Hi Sindy,Hi Sindy,
Unfortunatly I was not successful to have my internet access working.
I decide to try with OpenWRT giving me more flexibility and features.
Thanks a lot for your help.
Regards,
Nicolas
/interface bridge vlan
add vlan-ids=832 bridge=bridge tagged=sfp-sfpplus1,sfp-sfpplus2,sfp-sfpplus3,sfp-sfpplus4
/interface bridge set bridge vlan-filtering=yes
/interface ethernet switch chip rule
add switch=switch1 ports=sfp-sfpplus2 vlan-id=832 protocol=udp dst-port=67 new-dst-ports=sfp-sfpplus1 new-vlan-priority=6
add switch=switch1 ports=sfp-sfpplus2 vlan-id=832 new-dst-ports=sfp-sfpplus1
add switch=switch1 ports=sfp-sfpplus1 vlan-id=832 new-dst-ports=sfp-sfpplus2
add switch=switch1 ports=sfp-sfpplus4 vlan-id=832 protocol=udp dst-port=67 new-dst-ports=sfp-sfpplus3 new-vlan-priority=6
add switch=switch1 ports=sfp-sfpplus4 vlan-id=832 new-dst-ports=sfp-sfpplus3
add switch=switch1 ports=sfp-sfpplus3 vlan-id=832 new-dst-ports=sfp-sfpplus4
Yes, but only one of these bridges can (at least to date) outsource its job to the switch chip, so the other one would forward in software. That's why I've suggested the described solution with port isolation using switch chip rules. These rules do not slow down the processing and allow the frames to be forwarded only between the necessary pairs of ports.Does each bridge could act and handle each vlan 832 as separate vlan ?
Hi Sindy,Yes, it will work with hardware forwarding, hence at fiber speed. You probably have to make sure no traffic will leak between the two uplinks, or at least ensuring that should cause no harm.
So assuming the management interface of the CRS305 is ether1, the management IP subnet of the CRS305 is attached to bridge, and no VLAN tagging is used at Fortinet side for the management interface, the setup will be as follows:
I assume when you mention "DHCP request only", it actually means anything the DHCP client sends. Because in the DHCP vernacular, DHCPDISCOVER and DHCPREQUEST are two different messages. And there's no way to distinguish between these two messages by the fields the switch chip rules are able to match on.Code: Select all/interface bridge vlan add vlan-ids=832 bridge=bridge tagged=sfp-sfpplus1,sfp-sfpplus2,sfp-sfpplus3,sfp-sfpplus4 /interface bridge set bridge vlan-filtering=yes /interface ethernet switch chip rule add switch=switch1 ports=sfp-sfpplus2 vlan-id=832 protocol=udp dst-port=67 new-dst-ports=sfp-sfpplus1 new-vlan-priority=6 add switch=switch1 ports=sfp-sfpplus2 vlan-id=832 new-dst-ports=sfp-sfpplus1 add switch=switch1 ports=sfp-sfpplus1 vlan-id=832 new-dst-ports=sfp-sfpplus2 add switch=switch1 ports=sfp-sfpplus4 vlan-id=832 protocol=udp dst-port=67 new-dst-ports=sfp-sfpplus3 new-vlan-priority=6 add switch=switch1 ports=sfp-sfpplus4 vlan-id=832 new-dst-ports=sfp-sfpplus3 add switch=switch1 ports=sfp-sfpplus3 vlan-id=832 new-dst-ports=sfp-sfpplus4
A forum like this is the second best thing to an actual support for the users, but if Mikrotik was to provide this amount of support directly, you would have to pay a lot for a support contract or the equipment would have to cost much more (look at Cisco, there's both, the equipment is expensive and the access to service desk is paid as well).Mikrotik support is rock !
Yes, both.Both could be done or not ?
But doing so limits the total uplink bandwidth to the one of a single port.Yes you point a good optimization using one link only with the Firewall from the CRS305 and carry inside both wan (vlan 11 and 12 on the firewall side but vlan 832 for both wan with both ONU)
Correct. Switch rules can do multiple "actions" simultaneously - change CoS, change VID, and restrict the list of permitted egress ports.To be sure, inside the same switch rule ingress traffic on the firewall port I can detect a specific vlan (like 11 or 12), apply CoS 6 to DHCP trafic and translate vlan id to 832 and forward trafic for vlan 11 to port 2 and vlan 12 to port 3, all other non DHCP traffic translate vlan id to 832 and forward trafic for vlan 11 to port 2 and vlan 12 to port 3, is it correct ?
/interface bridge
add admin-mac=18:FD:74:00:44:70 auto-mac=no comment="Internet ONUs" ingress-filtering=no name=bridge vlan-filtering=yes
add comment="Mgt FortiSwitch PoE" name=mgt
/interface ethernet
set [ find default-name=ether1 ] comment=FortiSwitch
set [ find default-name=sfp-sfpplus1 ] comment=FortiGate
set [ find default-name=sfp-sfpplus2 ] comment="Not used"
set [ find default-name=sfp-sfpplus3 ] comment=Orange
set [ find default-name=sfp-sfpplus4 ] comment=Orange-Pro
/interface bridge port
add bridge=bridge comment=defconf ingress-filtering=no interface=sfp-sfpplus1
add bridge=bridge comment=defconf ingress-filtering=no interface=sfp-sfpplus2
add bridge=bridge comment=defconf ingress-filtering=no interface=sfp-sfpplus3
add bridge=bridge comment=defconf ingress-filtering=no interface=sfp-sfpplus4
add bridge=mgt ingress-filtering=no interface=ether1
/interface bridge vlan
add bridge=bridge tagged=sfp-sfpplus1 vlan-ids=111
add bridge=bridge tagged=sfp-sfpplus1 vlan-ids=112
add bridge=bridge tagged=sfp-sfpplus3,sfp-sfpplus4 vlan-ids=832
/interface ethernet switch rule
add comment="Intercept DHCPv4 Request Orange Vlan 111 port sfp1 and forward with vlan 832 and CoS 6 to port sfp3" dst-port=67 new-dst-ports=sfp-sfpplus3 new-vlan-id=832 new-vlan-priority=6 ports=sfp-sfpplus1 protocol=udp switch=switch1 vlan-id=111
add comment="Intercept All Trafic Orange Vlan 111 port sfp1 and forward with vlan 832 and CoS 6 to port sfp3" new-dst-ports=sfp-sfpplus3 new-vlan-id=832 ports=sfp-sfpplus1 switch=switch1 vlan-id=111
add comment="Intercept DHCPv4 Request Orange Pro Vlan 112 port sfp1 and forward with vlan 832 and CoS 6 to port sfp4" dst-port=67 new-dst-ports=sfp-sfpplus4 new-vlan-id=832 new-vlan-priority=6 ports=sfp-sfpplus1 protocol=udp switch=switch1 vlan-id=112
add comment="Intercept All Trafic Orange Pro Vlan 112 port sfp1 and forward with vlan 832 and CoS 6 to port sfp4" new-dst-ports=sfp-sfpplus4 new-vlan-id=832 ports=sfp-sfpplus1 switch=switch1 vlan-id=112
add comment="Intercept All Trafic back from Orange Vlan 832 port sfp3 and forward with vlan 111 to port sfp1" new-dst-ports=sfp-sfpplus1 new-vlan-id=111 ports=sfp-sfpplus3 switch=switch1 vlan-id=832
add comment="Intercept All Trafic back from Orange Pro Vlan 832 port sfp4 and forward with vlan 112 to port sfp1" new-dst-ports=sfp-sfpplus1 new-vlan-id=112 ports=sfp-sfpplus4 switch=switch1 vlan-id=832
/ip dhcp-client
add interface=mgt
/ip service
set telnet disabled=yes
set ftp disabled=yes
set www disabled=yes
set ssh port=1122
set www-ssl certificate=crs-chanet disabled=no port=11443
set api disabled=yes
set winbox disabled=yes
set api-ssl disabled=yes
/system package update
set channel=testing
/system routerboard settings
set boot-os=router-os
/tool sniffer
set filter-interface=sfp-sfpplus1 streaming-enabled=yes streaming-server=10.255.16.222
# may/18/2022 21:51:27 by RouterOS 7.3beta40
# software id = 3WGM-VXU2
#
# model = CRS305-1G-4S+
# serial number = HCB08F5PJ0G
/interface bridge
add admin-mac=18:FD:74:00:44:70 auto-mac=no comment="Internet ONUs" ingress-filtering=no name=bridge vlan-filtering=yes
add comment="Mgt FortiSwitch PoE" name=mgt
/interface ethernet
set [ find default-name=ether1 ] comment=FortiSwitch
set [ find default-name=sfp-sfpplus1 ] comment="FortiGate Orange & Orange-Pro"
set [ find default-name=sfp-sfpplus2 ] auto-negotiation=no comment="ONU Orange" speed=2.5Gbps
set [ find default-name=sfp-sfpplus3 ] comment="Not Used"
set [ find default-name=sfp-sfpplus4 ] auto-negotiation=no comment="ONU Orange-Pro" speed=2.5Gbps
/interface lte apn
set [ find default=yes ] ip-type=ipv4 use-network-apn=no
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/port
set 0 name=serial0
/interface bridge port
add bridge=bridge comment=defconf ingress-filtering=no interface=sfp-sfpplus1
add bridge=bridge comment=defconf ingress-filtering=no interface=sfp-sfpplus2
add bridge=bridge comment=defconf ingress-filtering=no interface=sfp-sfpplus3
add bridge=bridge comment=defconf ingress-filtering=no interface=sfp-sfpplus4
add bridge=mgt ingress-filtering=no interface=ether1
/ip settings
set max-neighbor-entries=8192
/interface bridge vlan
add bridge=bridge comment="Orange & Orange-Pro ONUs" tagged=sfp-sfpplus2,sfp-sfpplus4 vlan-ids=832
add bridge=bridge comment="FortiGate Orange" tagged=sfp-sfpplus1 vlan-ids=111
add bridge=bridge comment="FortiGate Orange-Pro" tagged=sfp-sfpplus1 vlan-ids=112
/interface ethernet switch rule
add comment="Intercept Orange DHCPv4 Request with vlan 111 on port sfp1 and forward it with vlan 832 and CoS 6 to port sfp2" dst-port=67 new-dst-ports=sfp-sfpplus2 new-vlan-id=832 new-vlan-priority=6 ports=sfp-sfpplus1 protocol=udp switch=switch1 vlan-id=111
add comment="Intercept All Orange Trafic with vlan 111 on port sfp1 and forward it with vlan 832 and CoS 6 to port sfp2" new-dst-ports=sfp-sfpplus2 new-vlan-id=832 ports=sfp-sfpplus1 switch=switch1 vlan-id=111
add comment="Intercept All Orange Trafic back from Vlan 832 on port sfp2 and forward it to port sfp1 with vlan 111" new-dst-ports=sfp-sfpplus1 new-vlan-id=111 ports=sfp-sfpplus2 switch=switch1 vlan-id=832
add comment="Intercept Orange-Pro DHCPv4 Request with vlan 112 on port sfp1 and forward it with vlan 832 and CoS 6 to port sfp4" dst-port=67 new-dst-ports=sfp-sfpplus4 new-vlan-id=832 new-vlan-priority=6 ports=sfp-sfpplus1 protocol=udp switch=switch1 vlan-id=112
add comment="Intercept All Orange-Pro Trafic with vlan 112 on port sfp1 and forward it with vlan 832 and CoS 6 to port sfp4" new-dst-ports=sfp-sfpplus4 new-vlan-id=832 ports=sfp-sfpplus1 switch=switch1 vlan-id=112
add comment="Intercept All Orange-Pro Trafic back from Vlan 832 on port sfp4 and forward it to port sfp1 with vlan 112" new-dst-ports=sfp-sfpplus1 new-vlan-id=112 ports=sfp-sfpplus4 switch=switch1 vlan-id=832
/interface ovpn-server server
set auth=sha1,md5
/ip dhcp-client
add interface=mgt
/ip service
set telnet disabled=yes
set ftp disabled=yes
set www disabled=yes
set ssh port=1122
set www-ssl certificate=crs-chanet disabled=no port=11443
set api disabled=yes
set winbox disabled=yes
set api-ssl disabled=yes
/system clock
set time-zone-name=Europe/Paris
/system identity
set name=CRS305
/system package update
set channel=testing
/system routerboard settings
set boot-os=router-os
/tool sniffer
set filter-interface=sfp-sfpplus1 streaming-enabled=yes streaming-server=10.255.16.222
Flags: R - RUNNING; S - SLAVE
Columns: NAME, RX-BYTE, TX-BYTE, RX-PACKET, TX-PACKET, RX-DROP, TX-DROP, TX-QUEUE-DROP, RX-ERROR, TX-ERROR
# NAME RX-BYTE TX-BYTE RX-PACKET TX-PACKET RX-DROP TX-DROP TX-QUEUE-DROP RX-ERROR TX-ERROR
;;; FortiSwitch
0 RS ether1 898 096 2 251 346 3 696 2 421 0
;;; FortiGate Orange & Orange-Pro
1 RS sfp-sfpplus1 124 339 651 130 395 486 154 075 185 862 0
;;; ONU Orange
2 RS sfp-sfpplus2 114 841 217 118 956 857 172 224 146 234 0
;;; Not Used
3 S sfp-sfpplus3 0 0 0 0 0
;;; ONU Orange-Pro
4 RS sfp-sfpplus4 15 553 730 5 392 074 13 642 7 997 0
;;; Internet ONUs
5 R bridge 0 220 0 2 0 0 0 0 0
;;; Mgt FortiSwitch PoE
6 R mgt 854 066 2 240 803 3 542 2 411 0 0 0 0 0
Flags: R - RUNNING; S - SLAVE
Columns: NAME, RX-BYTE, TX-BYTE, RX-PACKET, TX-PACKET, RX-DROP, TX-DROP, TX-QUEUE-DROP, RX-ERROR, TX-ERROR
# NAME RX-BYTE TX-BYTE RX-PACKET TX-PACKET RX-DROP TX-DROP TX-QUEUE-DROP RX-ERROR TX-ERROR
;;; ONU Orange
2 RS sfp-sfpplus2 114 841 217 118 956 857 172 224 146 234 0
;;; ONU Orange-Pro
4 RS sfp-sfpplus4 15 553 730 5 392 074 13 642 7 997 0
add comment="Intercept Orange DHCPv4 Request with vlan 111 on port sfp1 and forward it with vlan 832 and CoS 6 to port sfp2" \
switch=switch1 ports=sfp-sfpplus1 vlan-id=111 protocol=udp dst-port=67 new-dst-ports=sfp-sfpplus2 new-vlan-id=832 new-vlan-priority=6
add comment="Intercept All Orange Trafic with vlan 111 on port sfp1 and forward it with vlan 832 and CoS 6 to port sfp2" \
switch=switch1 ports=sfp-sfpplus1 vlan-id=111 new-dst-ports=sfp-sfpplus2 new-vlan-id=832
add comment="Intercept All Orange Trafic back from Vlan 832 on port sfp2 and forward it to port sfp1 with vlan 111" \
switch=switch1 ports=sfp-sfpplus2 vlan-id=832 new-dst-ports=sfp-sfpplus1 new-vlan-id=111
add comment="Intercept Orange-Pro DHCPv4 Request with vlan 112 on port sfp1 and forward it with vlan 832 and CoS 6 to port sfp4" \
switch=switch1 ports=sfp-sfpplus1 vlan-id=112 protocol=udp dst-port=67 new-dst-ports=sfp-sfpplus4 new-vlan-id=832 new-vlan-priority=6
add comment="Intercept All Orange-Pro Trafic with vlan 112 on port sfp1 and forward it with vlan 832 and CoS 6 to port sfp4" \
switch=switch1 ports=sfp-sfpplus1 vlan-id=112 new-dst-ports=sfp-sfpplus4 new-vlan-id=832
add comment="Intercept All Orange-Pro Trafic back from Vlan 832 on port sfp4 and forward it to port sfp1 with vlan 112" \
switch=switch1 ports=sfp-sfpplus4 vlan-id=832 new-dst-ports=sfp-sfpplus1 new-vlan-id=112[code]
Correct. I think you can sniff if you force copying of the ingress frames to the CPU, but you have to make sure the traffic volume is low in order not to overload the CPU.Switch Rules is not easy to troubleshoot, no stats no monitoring and If I am correct when there is a Switch Rule matching I cannot have packet capture with streaming ?
What I had in mind was a single common network at Orange side. In the beginning, it wasn't clear that both uplinks were provided by the same ISP, so I did not take these L2 issues into consideration. If you can use Architecture 2 and connect something else than the firewall to sfp-sfpplus1, you should see immediately whether the issue is related to the common MAC address at your end of both uplinks or not.As you mention there is no commun part for both Orange Fiber Internet access (Orange and Orange Pro), even when they share the same cable between the CRS305 and the Firewall they are each on different vlans.
No, you cannot - there is no switch chip rule that can change the source MAC address of a frame, let alone that it could change it inside the payload of the DHCP packets where it is probably used as the Client ID field. But if you configure the two WAN ports of the firewall as separate IP interfaces, they should use distinct MAC addresses automatically. Regarding Architecture 1, there may be a possibility to assign a MAC address to the interface in VLAN 112 manually, but I don't know Fortigate enough to tell you whether it is indeed possible and if yes, how exactly to configure it.On the CRS305 can I force a different MAC when I will use sfpplus3 to carry the second wan to the Firewall ?