Community discussions

MikroTik App
 
User avatar
BrateloSlava
Member Candidate
Member Candidate
Topic Author
Posts: 168
Joined: Mon Aug 09, 2021 10:33 am
Location: Ukraine, Kharkiv

Switch CRS112-8P-4S as a media converter

Wed Sep 14, 2022 5:41 pm

The connection diagram looks like this:
  • two optical connections to two Internet providers are connected to the SFP9 and SFP10 ports of the CRS112-8P-4S switch
  • CRS112-8P-4S has two bridges - SFP9+Ether1=Bridge-ISP1, SFP10+Ether2+Ether3=Bridge-ISP2
_Office.png

That is, CRS112-8P-4S is used as a media converter.

Configuration rules:
/interface bridge add name=bridge-ISP1
/interface bridge add name=bridge-ISP2
/interface ethernet set [ find default-name=ether1 ] comment="ISP1 -> 4011 port 1" poe-out=off
/interface ethernet set [ find default-name=ether2 ] comment="ISP2 -> 4011 port 2" poe-out=off
/interface ethernet set [ find default-name=ether3 ] comment="ISP2 -> 4011 port 3 (IPTV)" poe-out=off
/interface ethernet set [ find default-name=ether8 ] comment="-> 4011 port 4"
/interface ethernet set [ find default-name=sfp9 ] comment=ISP1
/interface ethernet set [ find default-name=sfp10 ] comment=ISP2

/interface bridge port add bridge=bridge-ISP1 ingress-filtering=no interface=ether1 trusted=yes
/interface bridge port add bridge=bridge-ISP1 ingress-filtering=no interface=sfp9 trusted=yes

/interface bridge port add bridge=bridge-ISP2 ingress-filtering=no interface=ether2
/interface bridge port add bridge=bridge-ISP2 ingress-filtering=no interface=sfp10
/interface bridge port add bridge=bridge-ISP2 ingress-filtering=no interface=ether3
Firewall rules:
/interface list add name=LAN
/interface list add name=WAN

/interface list member add interface=ether8 list=LAN
/interface list member add interface=bridge-ISP1 list=WAN
/interface list member add interface=bridge-ISP2 list=WAN


/ip firewall filter add action=accept chain=input comment="Handle (input) already established, related, untracked connections" connection-state=established,related,untracked
/ip firewall filter add action=accept chain=forward comment="Handle (forward) already established, related, untracked connections" connection-state=established,related,untracked
/ip firewall filter add action=accept chain=output comment="Handle (output) already established, related connections" connection-state=established,related
/ip firewall filter add action=accept chain=input comment="DHCP request ISP1" in-interface=bridge-ISP1 port=67,68 protocol=udp
/ip firewall filter add action=accept chain=forward in-interface=bridge-ISP1 port=67,68 protocol=udp
/ip firewall filter add action=accept chain=input comment="<- LAN ->" in-interface-list=LAN
/ip firewall filter add action=accept chain=forward in-interface-list=LAN
/ip firewall filter add action=accept chain=output out-interface-list=LAN
/ip firewall filter add action=drop chain=input comment="Drop all from WAN" in-interface-list=WAN
/ip firewall filter add action=log chain=input comment="Log everything else"
/ip firewall filter add action=log chain=forward
/ip firewall filter add action=log chain=output
All ports included in bridges are in hardware offload mode.

Question: how to properly configure such a configuration? It is embarrassing, that already at a speed of 50-60 Mbps, the CPU load is about 70%.

Screenshot_profile.png
You do not have the required permissions to view the files attached to this post.
 
User avatar
mkx
Forum Guru
Forum Guru
Posts: 11433
Joined: Thu Mar 03, 2016 10:23 pm

Re: Switch CRS112-8P-4S as a media converter

Wed Sep 14, 2022 8:05 pm

Only one bridge (per seitch chip) can be hardware offloaded, others are handled entirely by CPU.

What you could do is configure all ports into single bridge but separate them into VLANs using switch chip configuration. It is fine to make all ports members of a VLAN access ports (=untagged), which in a sense partitions a switch while alliwing it to do it in hardware.
 
User avatar
BrateloSlava
Member Candidate
Member Candidate
Topic Author
Posts: 168
Joined: Mon Aug 09, 2021 10:33 am
Location: Ukraine, Kharkiv

Re: Switch CRS112-8P-4S as a media converter

Wed Sep 14, 2022 8:12 pm

Screenshot_HO.png
It is possible, that this is a mistake.
You do not have the required permissions to view the files attached to this post.
 
User avatar
mkx
Forum Guru
Forum Guru
Posts: 11433
Joined: Thu Mar 03, 2016 10:23 pm

Re: Switch CRS112-8P-4S as a media converter

Wed Sep 14, 2022 8:33 pm

It is possible, that this is a mistake.

It's your device, it's your problem ... I'm not going to argue with you.
 
User avatar
BrateloSlava
Member Candidate
Member Candidate
Topic Author
Posts: 168
Joined: Mon Aug 09, 2021 10:33 am
Location: Ukraine, Kharkiv

Re: Switch CRS112-8P-4S as a media converter

Wed Sep 14, 2022 9:23 pm

It's your device, it's your problem ... I'm not going to argue with you.

I didn't mean that your answer is a mistake. I meant that ROS 7.5 contains a bug.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Switch CRS112-8P-4S as a media converter

Wed Sep 14, 2022 9:26 pm

It is possible, that this is a mistake.
It might be an error of the indeed Winbox display indeed, but the manual for CRS1xx/2xx devices states the following:

Multiple switch groups
The CRS1xx/2xx series switches allow you to use multiple bridges with hardware offloading, this allows you to easily isolate multiple switch groups. This can be done by simply creating multiple bridges and enabling hardware offloading.


Given this, I'd rather look why it behaves different than expected than suspect display errors. I can imagine some broadcast traffic to bother the CPU, so it might make sense to prevent any traffic from the "transit" interfaces from reaching the CPU using switch chip rules (called ACL rules in CRS1xx/2xx case). But you have to keep in mind to allow all traffic, including broadcast frames, to reach the CPU from an interface used for management.

Once you start using the ACL rules, you may as well use a single bridge and use the ACL rules to allow ingress frames from ether1 only to egress via sfp9 and vice versa, and ingress frames from ether2 and 3 to egress via SFP10 and vice versa.

What I don't like about the setup is that STP is permitted on the bridge - depending on how paranoid the ISP admins are, you may or may not break their own spanning tree topology by making your CRS a root bridge. So I'd rather disable it on the bridge(s) completely.
 
User avatar
BrateloSlava
Member Candidate
Member Candidate
Topic Author
Posts: 168
Joined: Mon Aug 09, 2021 10:33 am
Location: Ukraine, Kharkiv

Re: Switch CRS112-8P-4S as a media converter

Wed Sep 14, 2022 11:18 pm

... What I don't like about the setup is that STP is permitted on the bridge - depending on how paranoid the ISP admins are, you may or may not break their own spanning tree topology by making your CRS a root bridge. So I'd rather disable it on the bridge(s) completely.

The current config file looks like this:
# sep/14/2022 20:37:18 by RouterOS 7.5
# software id = SZII-F003
#
# model = CRS112-8P-4S
# serial number = 
/interface bridge add name=bridge-LAN
/interface bridge add name=bridge-Maxnet
/interface bridge add name=bridge-Triolan
/interface ethernet set [ find default-name=ether1 ] comment="Maxnet -> 4011 port 1" poe-out=off
/interface ethernet set [ find default-name=ether2 ] comment="Triolan -> 4011 port 2" poe-out=off
/interface ethernet set [ find default-name=ether3 ] comment="Triolan -> 4011 port 3 (IPTV)" poe-out=off
/interface ethernet set [ find default-name=ether8 ] comment="-> 4011 port 4"
/interface ethernet set [ find default-name=sfp9 ] comment=Maxnet
/interface ethernet set [ find default-name=sfp10 ] comment=Triolan
/interface list add name=LAN
/interface list add name=WAN
/interface bridge port add bridge=bridge-Maxnet ingress-filtering=no interface=ether1 trusted=yes
/interface bridge port add bridge=bridge-Maxnet ingress-filtering=no interface=sfp9 trusted=yes
/interface bridge port add bridge=bridge-Triolan ingress-filtering=no interface=ether2
/interface bridge port add bridge=bridge-Triolan ingress-filtering=no interface=sfp10
/interface bridge port add bridge=bridge-Triolan ingress-filtering=no interface=ether3
/interface bridge port add bridge=bridge-LAN interface=ether8
/interface bridge port add bridge=bridge-LAN interface=sfp12
/ip firewall connection tracking set icmp-timeout=30s tcp-close-wait-timeout=1m tcp-established-timeout=1h tcp-fin-wait-timeout=2m tcp-last-ack-timeout=30s tcp-syn-received-timeout=1m tcp-syn-sent-timeout=2m tcp-time-wait-timeout=2m udp-stream-timeout=2m udp-timeout=30s
/ip neighbor discovery-settings set discover-interface-list=none
/ip settings set tcp-syncookies=yes
/ipv6 settings set disable-ipv6=yes
/interface list member add interface=bridge-Maxnet list=WAN
/interface list member add interface=bridge-Triolan list=WAN
/interface list member add interface=bridge-LAN list=LAN
/ip address add address=172.22.0.231/24 interface=bridge-LAN network=172.22.0.0
/ip dns set cache-max-ttl=5m servers=172.22.0.254
/ip firewall raw add action=accept chain=prerouting comment="DHCP request Maxnet" in-interface=bridge-Maxnet port=67,68 protocol=udp
/ip firewall raw add action=drop chain=prerouting comment="Drop all from WAN" in-interface-list=WAN
/ip firewall service-port set tftp disabled=yes
/ip firewall service-port set irc disabled=yes
/ip firewall service-port set h323 disabled=yes
/ip firewall service-port set sip disabled=yes
/ip firewall service-port set pptp disabled=yes
/ip firewall service-port set dccp disabled=yes
/ip firewall service-port set sctp disabled=yes
/ip route add check-gateway=arp disabled=no dst-address=0.0.0.0/0 gateway=172.22.0.254
/ip service set telnet address=172.16.0.0/12
/ip service set ftp address=172.16.0.0/12
/ip service set www disabled=yes
/ip service set ssh address=172.16.0.0/12
/ip service set api disabled=yes
/ip service set api-ssl disabled=yes
/ip ssh set strong-crypto=yes
/snmp set contact=Admin enabled=yes location="MO IT" trap-version=2
/system clock set time-zone-name=Europe/Kiev
/system identity set name=rt-mo-optic
/system ntp client set enabled=yes
/system ntp client servers add address=172.22.0.254
/system routerboard settings set auto-upgrade=yes
/tool bandwidth-server set enabled=no
/tool mac-server set allowed-interface-list=LAN
/tool mac-server mac-winbox set allowed-interface-list=LAN

  1. Switching bridges from mode "RSTP" to mode "none" only worsened the situation: the CPU utilization increased to 95-100%.
  2. Do I understand correctly, that I need to remove all current bridges, and then create a new bridge and add all necessary ports to it. And then use this instruction
Last edited by BrateloSlava on Thu Sep 15, 2022 2:06 pm, edited 1 time in total.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Switch CRS112-8P-4S as a media converter

Wed Sep 14, 2022 11:46 pm

  1. Switching bridges from mode "RSTP" to mode "none" only worsened the situation: the CPU utilization increased to 95-100%.
That's strange if the partitioning of the switch works; if it doesn't, I'm afraid the ISP admins are scratching their heads what it was as you may have interconnected their networks together.

  1. Do I understand correctly, that I need to remove all current bridges, and then create a new bridge and add all necessary ports to it. And then use this instruction
If the switch chip partitioning by configuring multiple bridges fails, then yes, this other approach may be the solution. Except that this instruction describes a more usual case where the three groups (communities) share the uplink ports (traffic between any uplink port and any downlink port is permitted, and so is traffic between any two downlink ports in the same group, only traffic between downlink ports in different groups is forbidden), so you have to adjust the isolation profiles to match your needs, i.e. you would put ether1 and sfp9 into community 0 (isolation-leakage-profile-override=2) and ether2, ether3, and sfp10 into community 1 (isolation-leakage-profile-override=3). However, the example does not say what happens to other ports and what happens to the CPU-facing port when you set these, so expect some sparks until you grasp it. And the goal remains to prevent traffic from each group from leaking elsewhere, not only to the other group but also to the CPU, and at the same time keep the device manageable through another port. So attaching a management subnet to an Ethernet interface that is not a member port of any bridge is the first thing to do before you even start redoing the rest.

If I were you, I would first check whether the traffic indeed leaks between the currently configured bridges by placing the switch on the table and connecting two PCs to it to see whether they can talk to each other when connected to two ports in the same bridge (they should) and to two ports on different bridges (they should not). Of course with copper ports added to each bridge unless you have copper SFPs. And only if it turns out that the "dual bridge" approach indeed leaks the traffic between the bridges, I'd move to the other approach.
 
User avatar
BrateloSlava
Member Candidate
Member Candidate
Topic Author
Posts: 168
Joined: Mon Aug 09, 2021 10:33 am
Location: Ukraine, Kharkiv

Re: Switch CRS112-8P-4S as a media converter

Wed Sep 21, 2022 2:00 pm

The results of the experiments are negative.

Note 1: I strongly recommend, that you have a console cable for recovery at hand, when experimenting with setting up switch chips.

Current (initial) configuration: 3 bridges. The first is provider 1 (2 ports), the second is provider 2 (3 ports), the third is the local network (2 ports).

All bridge ports are in hardware offload mode. As described in the documentation.

The problem with the initial configuration is a high CPU load with constant traffic at a speed of 50+ Mbps.

What has been tested:
  1. Complete reset of the configuration, reconfiguring the current configuration with three bridges. All operations were performed from the command line.
  2. Full configuration reset, ROS 7.5 installation via Netinstall , configuration setting in the two-bridge option (only bridges for providers). Configuration was done through the console port.
  3. Complete configuration reset, configuration setting in the single bridge option. Used the division of traffic between ports using a switching chip. Configuration was done through the console port.
The result of all three tests is the same - no decrease in the load on the central processor was noticed.

Note 2: I would like to note that all options were configured as follows: disconnecting optical cables from providers, complete configuration reset, configuration, reboot, connecting cables from providers, reboot.
CPU load monitoring started 15 minutes after the last reboot.

When using the third setting option (using a switching chip), the following materials were used:
https://help.mikrotik.com/docs/pages/vi ... -Isolation
https://help.mikrotik.com/docs/pages/vi ... =103841835
https://wiki.mikrotik.com/wiki/Manual:C ... on/Leakage
https://www.youtube.com/watch?v=KhPq222VpFM
https://citraweb.com/artikel_lihat.php?id=254

Example settings for the third option:
# Create new bridge

/interface bridge add name=bridge1 protocol-mode=none
           
/interface bridge port add bridge=bridge1 interface=sfp9 hw=yes
/interface bridge port add bridge=bridge1 interface=ether1 hw=yes
/interface bridge port add bridge=bridge1 interface=sfp10 hw=yes
/interface bridge port add bridge=bridge1 interface=ether2 hw=yes
/interface bridge port add bridge=bridge1 interface=ether3 hw=yes

# Maxnet
/interface ethernet switch port
set sfp9 isolation-leakage-profile-override=2
set ether1 isolation-leakage-profile-override=2
/

/interface ethernet switch port-isolation
add port-profile=2 ports=sfp9,ether1 type=dst
/

# Triolan
/interface ethernet switch port
set sfp10 isolation-leakage-profile-override=3
set ether2 isolation-leakage-profile-override=3
set ether3 isolation-leakage-profile-override=3
/

/interface ethernet switch port-isolation
add port-profile=3 ports=sfp10,ether2,ether3 type=dst
/

Note 3: Downgrading via Netinstall to firmware version 6.49.6 Stable has not been tested.
Note 4: after disconnecting the cables from the providers and completely resetting the settings, there is a constant CPU load of 10%. In this case, only the cable between the CRS112-8P-4S and the laptop was connected. Observation was made using Winbox.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Switch CRS112-8P-4S as a media converter

Wed Sep 21, 2022 4:21 pm

What happens if you run /tool sniffer quick interface=bridgeX (where bridgeX is one of the ISP bridges)? I mean, either there's a bug or the CPU load is caused by too many packets reaching the CPU as I've suggested initially, and the sniffer should show you how many such packets there actually are.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11982
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Switch CRS112-8P-4S as a media converter

Wed Sep 21, 2022 6:34 pm

/ip settings set tcp-syncookies=yes
why?

why firewall rules?

you use the switch as router for act as firewall???

No one single packet must reach the cpu, all you firewall rules must be useless if the traffic must go only from ISP to the RB4011

Is obvious than if you use the device as firewall all traffic pass though CPU before go anywhere....


Why use two cable to connect same ISP on RB4011???
Probably the problem is also on other side that the cable are on loop....

Why another cable to connect back the RB4011??? 4 cable for do the work of two???...
 
User avatar
BrateloSlava
Member Candidate
Member Candidate
Topic Author
Posts: 168
Joined: Mon Aug 09, 2021 10:33 am
Location: Ukraine, Kharkiv

Re: Switch CRS112-8P-4S as a media converter

Wed Sep 21, 2022 11:12 pm

Why use two cable to connect same ISP on RB4011???
Probably the problem is also on other side that the cable are on loop....
Why another cable to connect back the RB4011??? 4 cable for do the work of two???...
Two providers. The first - provides one IP address, the second - two. Two addresses - for two tasks. Therefore, two interfaces for 4011 from the second provider are connected.
Screenshot_4011.png

As for the rules of the firewall. I do not like attempts to explore the MAC address or conduct another incomprehensible and unknown activity to me. Therefore, I wanted to close the incoming chain.

The third bridge is for connecting from the local network. For the purpose of monitoring and tuning. This bridge also has two interfaces, because sometimes equipment is replaced in the local network and I need to be able to connect via optics or twisted pair.

I understand that all these settings can be done differently, using VLANs. And then only one cable is required between the switch and the router. To be honest - it seemed to me that a scheme with several bridges would be easier for the switch to process. And the load on the switch's CPU will be lower.

In addition, I tried to configure a variant without any firewall rules, on a single bridge, using a switching chip. The result did not please me. It's possible that I'm setting something up wrong.
You do not have the required permissions to view the files attached to this post.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11982
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Switch CRS112-8P-4S as a media converter

Thu Sep 22, 2022 12:27 am

Is possble to have more than one IP on same interface...

I have 4096 IPs from my upstream, but I do not need 4096 cables... :lol:

If you want use 8P as media converter, use it as media converter,
any operation than simply switching (is a switch....) cause high CPU load...

For use it as double media converter, and keeping sfp12 and ether8 (and also the other ports not mentioned) for other bridges, management, or other uses,
reset all configuration without default, when you do /export must not export any relevant.
/interface bridge
add name=bri-ISPs protocol-mode=none
add name=bri-LAN protocol-mode=none

/interface bridge port
add bridge=bri-ISPs interface=ether1 hw=yes
add bridge=bri-ISPs interface=ether2 hw=yes
add bridge=bri-ISPs interface=ether3 hw=yes
add bridge=bri-LAN interface=ether4 hw=no
add bridge=bri-LAN interface=ether5 hw=no
add bridge=bri-LAN interface=ether6 hw=no
add bridge=bri-LAN interface=ether7 hw=no
add bridge=bri-LAN interface=ether8 hw=no
add bridge=bri-ISPs interface=sfp9 hw=yes
add bridge=bri-ISPs interface=sfp10 hw=yes
add bridge=bri-LAN interface=sfp11 hw=no
add bridge=bri-LAN interface=sfp12 hw=no

/interface ethernet switch port-isolation
set ether1 forwarding-override=sfp9
set sfp9 forwarding-override=ether1
set ether2 forwarding-override=ether3,sfp10
set ether3 forwarding-override=ether2,sfp10
set sfp10 forwarding-override=ether2,ether3

/ip address
add address=192.168.88.2/24 interface=bri-LAN

/ip dns
set servers=192.168.88.1

/ip route
add distance=1 gateway=192.168.88.1
And you have done.

No IP firewall reqired, because the device do not have any IP (except what is coming on ether8 from RB4011),
until you do not connect other internet sources.
 
tangent
Forum Guru
Forum Guru
Posts: 1351
Joined: Thu Jul 01, 2021 3:15 pm
Contact:

Re: Switch CRS112-8P-4S as a media converter

Thu Sep 22, 2022 1:10 am

Bingo. If you want line-rate switching, you cannot do anything to the packets that forces them to cross the CPU. That includes firewalling, but it isn’t limited to it. The packets have to stay on the switch chip.

This series has ACL rules that work purely at the switch chip level, but they’re less powerful than RouterOS’s full firewall.
 
User avatar
BrateloSlava
Member Candidate
Member Candidate
Topic Author
Posts: 168
Joined: Mon Aug 09, 2021 10:33 am
Location: Ukraine, Kharkiv

Re: Switch CRS112-8P-4S as a media converter

Thu Sep 22, 2022 8:47 am

/interface ethernet switch port-isolation
set ether1 forwarding-override=sfp9
set sfp9 forwarding-override=ether1
set ether2 forwarding-override=ether3,sfp10
set ether3 forwarding-override=ether2,sfp10
set sfp10 forwarding-override=ether2,ether3
The main idea is clear. Although these port isolation rules do not apply to this switch model. I described above, in paragraph 3, the rules, that I used to work on the switch chip. In this case, no firewall rules were used. However, the load on the CPU was still high. I will try, in the coming days, to once again experiment with working through the switching chip and post more complete results.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Switch CRS112-8P-4S as a media converter

Thu Sep 22, 2022 9:58 am

Is possble to have more than one IP on same interface...

I have 4096 IPs from my upstream, but I do not need 4096 cables... :lol:
On a single cable, you still have the bandwidth of a single cable (for all of those 4096 IP addresses together). So yes, where a single ISP fiber provides two IP addresses, maybe they could be attached to the same interface at the 4011, but I can see nothing strange on using another physical interface on the 4011 for the second ISP. That router can handle more than 1 Gbit/s per direction.

The whole idea @BrateloSlava wants to implement is correct, the only issue is where the load of the CRS CPU comes from.

@BrateloSlava, instead of running /tool sniffer, it is probably more useful to add the following bridge filter rules:
interface bridge filter add chain=input in-bridge=bridge-name dst-mac-address=ff:ff:ff:ff:ff:ff/ff:ff:ff:ff:ff:ff action=passthrough
interface bridge filter add chain=input in-bridge=bridge-name action=passthrough


and then run
/interface bridge filter print stats interval=10s

We are interested in the two subsequent outputs, to see how many packets and bytes got caught by each of the two rules during 10 seconds.

The idea is that only frames that reach the CPU port of the switch chip will get caught by one of these rules, so you'll see the amount of that traffic, and you'll also see how much of it is broadcast (presumably all of it) and how much is unicast (to the device's own MAC address, presumably none). Traffic between the SFP and the copper port will not reach the CPU so it won't get caught by these rules.

Of course, these rules will increase the CPU load even more, but it's necessary to do this test to understand whether the root cause of the high CPU load is too much broadcast traffic or a bug in 7.x. But to keep the CPU load as low as possible, I'd recommend to use the rules for just one bridge at a time.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11982
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Switch CRS112-8P-4S as a media converter

Thu Sep 22, 2022 4:13 pm

Is possble to have more than one IP on same interface...

I have 4096 IPs from my upstream, but I do not need 4096 cables... :lol:
On a single cable, you still have the bandwidth of a single cable (for all of those 4096 IP addresses together). So yes, where a single ISP fiber provides two IP addresses, maybe they could be attached to the same interface at the 4011, but I can see nothing strange on using another physical interface on the 4011 for the second ISP. That router can handle more than 1 Gbit/s per direction.
Yes, but I write "interface", and generic "cables", not "ethernet cable".

CRS112-8P-4S support only SFP and two Gb cables do not increase the total speed.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11982
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Switch CRS112-8P-4S as a media converter

Thu Sep 22, 2022 4:20 pm

@BrateloSlava
these port isolation rules do not apply to this switch model
Sorry
 
User avatar
BrateloSlava
Member Candidate
Member Candidate
Topic Author
Posts: 168
Joined: Mon Aug 09, 2021 10:33 am
Location: Ukraine, Kharkiv

Re: Switch CRS112-8P-4S as a media converter

Thu Sep 22, 2022 4:39 pm

@BrateloSlava, instead of running /tool sniffer, it is probably more useful to add the following bridge filter rules:
interface bridge filter add chain=input in-bridge=bridge-name dst-mac-address=ff:ff:ff:ff:ff:ff/ff:ff:ff:ff:ff:ff action=passthrough
interface bridge filter add chain=input in-bridge=bridge-name action=passthrough


and then run
/interface bridge filter print stats interval=10s
@sindy

Rules are added to both bridges that are connected to two providers.

Bridge ISP1
# sep/22/2022 17: 4:56 by RouterOS 7.5
# software id = SZII-F003
#
Flags: X, D - DYNAMIC
Columns: CHAIN, ACTION, BYTES, PACKETS
  CHAIN  ACTION        BYTES  PACKETS
  input  passthrough  329805     5757
  input  passthrough  829615    12742
X input  passthrough       0        0
X input  passthrough       0        0
Bridge ISP2
# sep/22/2022 17: 7: 0 by RouterOS 7.5
# software id = SZII-F003
#
Flags: X, D - DYNAMIC
Columns: CHAIN, ACTION, BYTES, PACKETS
  CHAIN  ACTION        BYTES  PACKETS
X input  passthrough       0        0
X input  passthrough       0        0
  input  passthrough  917076    19905
  input  passthrough  928290    20042
Current config
# sep/22/2022 17:10:52 by RouterOS 7.5
# software id = SZII-F003
#
# model = CRS112-8P-4S
/interface bridge
add name=bridge-LAN
add name=bridge-Maxnet
add name=bridge-Triolan
/interface ethernet
set [ find default-name=ether1 ] comment="Maxnet -> 4011 port 1" poe-out=off
set [ find default-name=ether2 ] comment="Triolan -> 4011 port 2" poe-out=off
set [ find default-name=ether3 ] comment="Triolan -> 4011 port 3 (IPTV)" poe-out=off
set [ find default-name=ether8 ] comment="-> 4011 port 4"
set [ find default-name=sfp9 ] comment=Maxnet
set [ find default-name=sfp10 ] comment=Triolan
/interface list
add name=LAN
add name=WAN
/interface bridge filter
add action=passthrough chain=input disabled=yes dst-mac-address=\
    FF:FF:FF:FF:FF:FF/FF:FF:FF:FF:FF:FF in-bridge=bridge-Maxnet
add action=passthrough chain=input disabled=yes in-bridge=bridge-Maxnet
add action=passthrough chain=input disabled=yes dst-mac-address=\
    FF:FF:FF:FF:FF:FF/FF:FF:FF:FF:FF:FF in-bridge=bridge-Triolan
add action=passthrough chain=input disabled=yes in-bridge=bridge-Triolan
/interface bridge port
add bridge=bridge-Maxnet ingress-filtering=no interface=ether1 trusted=yes
add bridge=bridge-Maxnet ingress-filtering=no interface=sfp9 trusted=yes
add bridge=bridge-Triolan ingress-filtering=no interface=ether2
add bridge=bridge-Triolan ingress-filtering=no interface=sfp10
add bridge=bridge-Triolan ingress-filtering=no interface=ether3
add bridge=bridge-LAN interface=ether8
add bridge=bridge-LAN interface=sfp12
/ip firewall connection tracking
set icmp-timeout=30s tcp-close-wait-timeout=1m tcp-established-timeout=1h \
    tcp-fin-wait-timeout=2m tcp-last-ack-timeout=30s \
    tcp-syn-received-timeout=1m tcp-syn-sent-timeout=2m \
    tcp-time-wait-timeout=2m udp-stream-timeout=2m udp-timeout=30s
/ip neighbor discovery-settings
set discover-interface-list=none
/ipv6 settings
set disable-ipv6=yes
/interface list member
add interface=ether7 list=LAN
add interface=bridge-Maxnet list=WAN
add interface=bridge-Triolan list=WAN
add interface=bridge-LAN list=LAN
/interface ovpn-server server
set auth=sha1,md5
/ip address
add address=172.22.0.231/24 interface=bridge-LAN network=172.22.0.0
/ip dns
set cache-max-ttl=5m servers=172.22.0.254
/ip firewall service-port
set tftp disabled=yes
set irc disabled=yes
set h323 disabled=yes
set sip disabled=yes
set pptp disabled=yes
set dccp disabled=yes
set sctp disabled=yes
/ip route
add check-gateway=arp disabled=no dst-address=0.0.0.0/0 gateway=172.22.0.254
/ip service
set www disabled=yes
set api disabled=yes
set api-ssl disabled=yes
/ip ssh
set strong-crypto=yes
/system clock
set time-zone-name=Europe/Kiev
/system identity
set name=rt-mo-optic
/system ntp client
set enabled=yes
/system ntp client servers
add address=172.22.0.254
/system routerboard settings
set auto-upgrade=yes
/tool bandwidth-server
set enabled=no
/tool mac-server
set allowed-interface-list=LAN
/tool mac-server mac-winbox
set allowed-interface-list=LAN
 
User avatar
Znevna
Forum Guru
Forum Guru
Posts: 1347
Joined: Mon Sep 23, 2019 1:04 pm

Re: Switch CRS112-8P-4S as a media converter

Thu Sep 22, 2022 5:15 pm

[...]Although these port isolation rules do not apply to this switch model. [...]
There's something special about this switch that prevents forwarding override from functioning? it's not mentioned anywhere. Please provide a link to where you got this "information" from.
As an addition to the rules posted above you'll have to override all the other ports too because otherwise they will forward traffic by default to all the other ports, including to those that you've set override rules (just that it's one way now because of that).
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11982
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Switch CRS112-8P-4S as a media converter

Thu Sep 22, 2022 5:56 pm

I have same CRS112-8P-4S, forward-override is not supported (at least on v6)
 
User avatar
Znevna
Forum Guru
Forum Guru
Posts: 1347
Joined: Mon Sep 23, 2019 1:04 pm

Re: Switch CRS112-8P-4S as a media converter

Thu Sep 22, 2022 5:57 pm

Sad then.
But it is mentioned in the brochure: https://i.mt.lv/cdn/product_files/CRS11 ... 200530.pdf
"It gives you all the basic functionality for a managed switch, plus more: allows to manage port-to-port forwarding,[...]"
So, what gives?
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Switch CRS112-8P-4S as a media converter

Thu Sep 22, 2022 6:10 pm

The absolute packet count is not as interesting as the packet rate, that's why I've said you should print the statistics in a 10 second interval and calculate the difference (i.e. the the packet rate per 10 s). But what bothers me even more is the number of non-broadcast packets reaching the bridge, but that's my own stupidity as I didn't take into account that there may be also multicast packets. So please replace the ff:ff:ff:ff:ff:ff/ff:ff:ff:ff:ff:ff by 01:00:00:00:00:00/01:00:00:00:00:00 in the rules, run the /interface bridge print stats interval=10s, and each time the numbers change, press the D key to take a snapshot. And then post three subsequent snapshots for each bridge.

Before you've edited the post it looked like 2000 frames per second which is quite a lot, but maybe the values were not 10 s part?
 
User avatar
BrateloSlava
Member Candidate
Member Candidate
Topic Author
Posts: 168
Joined: Mon Aug 09, 2021 10:33 am
Location: Ukraine, Kharkiv

Re: Switch CRS112-8P-4S as a media converter

Thu Sep 22, 2022 8:05 pm

@rextended
The documentation for this switch (CRS1xx/2xx series switches) has an ACL rules section. Is it possible that using these rules it is necessary to filter traffic between ports? Allow forwarding what you want and discard the rest.
 
User avatar
BrateloSlava
Member Candidate
Member Candidate
Topic Author
Posts: 168
Joined: Mon Aug 09, 2021 10:33 am
Location: Ukraine, Kharkiv

Re: Switch CRS112-8P-4S as a media converter

Thu Sep 22, 2022 8:18 pm

... press the D key to take a snapshot. And then post three subsequent snapshots for each bridge.
@sindy
New rule
/interface bridge filter
add action=passthrough chain=input disabled=yes dst-mac-address=01:00:00:00:00:00/01:00:00:00:00:00 in-bridge=bridge-Maxnet
add action=passthrough chain=input disabled=yes in-bridge=bridge-Maxnet
add action=passthrough chain=input disabled=yes dst-mac-address=01:00:00:00:00:00/01:00:00:00:00:00 in-bridge=bridge-Triolan
add action=passthrough chain=input disabled=yes in-bridge=bridge-Triolan
/
Result for ISP1
# sep/22/2022 20:11:23 by RouterOS 7.5
# software id = SZII-F003
#
Flags: X, D - DYNAMIC
Columns: CHAIN, ACTION, BYTES, PACKETS
  CHAIN  ACTION       BYTES  PACKETS
X input  passthrough      0        0
X input  passthrough      0        0
X input  passthrough      0        0
X input  passthrough      0        0
# sep/22/2022 20:11:30 by RouterOS 7.5
# software id = SZII-F003
#
Flags: X, D - DYNAMIC
Columns: CHAIN, ACTION, BYTES, PACKETS
  CHAIN  ACTION       BYTES  PACKETS
  input  passthrough   2440       36
  input  passthrough   1064       17
X input  passthrough      0        0
X input  passthrough      0        0
# sep/22/2022 20:11:35 by RouterOS 7.5
# software id = SZII-F003
#
Flags: X, D - DYNAMIC
Columns: CHAIN, ACTION, BYTES, PACKETS
  CHAIN  ACTION       BYTES  PACKETS
  input  passthrough   5126       78
  input  passthrough   3750       59
X input  passthrough      0        0
X input  passthrough      0        0
# sep/22/2022 20:11:46 by RouterOS 7.5
# software id = SZII-F003
#
Flags: X, D - DYNAMIC
Columns: CHAIN, ACTION, BYTES, PACKETS
  CHAIN  ACTION       BYTES  PACKETS
  input  passthrough   9629      151
  input  passthrough   8253      132
X input  passthrough      0        0
X input  passthrough      0        0
# sep/22/2022 20:11:50 by RouterOS 7.5
# software id = SZII-F003
#
Flags: X, D - DYNAMIC
Columns: CHAIN, ACTION, BYTES, PACKETS
  CHAIN  ACTION       BYTES  PACKETS
  input  passthrough  11136      174
  input  passthrough   9760      155
X input  passthrough      0        0
X input  passthrough      0        0
# sep/22/2022 20:11:54 by RouterOS 7.5
# software id = SZII-F003
#
Flags: X, D - DYNAMIC
Columns: CHAIN, ACTION, BYTES, PACKETS
  CHAIN  ACTION       BYTES  PACKETS
  input  passthrough  12460      196
  input  passthrough  11084      177
X input  passthrough      0        0
X input  passthrough      0        0

Result for ISP2
# sep/22/2022 20:13:54 by RouterOS 7.5
# software id = SZII-F003
#
Flags: X, D - DYNAMIC
Columns: CHAIN, ACTION, BYTES, PACKETS
  CHAIN  ACTION        BYTES  PACKETS
X input  passthrough       0        0
X input  passthrough       0        0
  input  passthrough  107432     2307
  input  passthrough   63557     1366
# sep/22/2022 20:14: 5 by RouterOS 7.5
# software id = SZII-F003
#
Flags: X, D - DYNAMIC
Columns: CHAIN, ACTION, BYTES, PACKETS
  CHAIN  ACTION        BYTES  PACKETS
X input  passthrough       0        0
X input  passthrough       0        0
  input  passthrough  582272    12542
  input  passthrough  538397    11601
# sep/22/2022 20:14:16 by RouterOS 7.5
# software id = SZII-F003
#
Flags: X, D - DYNAMIC
Columns: CHAIN, ACTION, BYTES, PACKETS
  CHAIN  ACTION         BYTES  PACKETS
X input  passthrough        0        0
X input  passthrough        0        0
  input  passthrough  1050646    22635
  input  passthrough  1006771    21694
# sep/22/2022 20:14:27 by RouterOS 7.5
# software id = SZII-F003
#
Flags: X, D - DYNAMIC
Columns: CHAIN, ACTION, BYTES, PACKETS
  CHAIN  ACTION         BYTES  PACKETS
X input  passthrough        0        0
X input  passthrough        0        0
  input  passthrough  1521795    32778
  input  passthrough  1477920    31837
# sep/22/2022 20:14:38 by RouterOS 7.5
# software id = SZII-F003
#
Flags: X, D - DYNAMIC
Columns: CHAIN, ACTION, BYTES, PACKETS
  CHAIN  ACTION         BYTES  PACKETS
X input  passthrough        0        0
X input  passthrough        0        0
  input  passthrough  1995693    42983
  input  passthrough  1951818    42042
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Switch CRS112-8P-4S as a media converter

Thu Sep 22, 2022 9:43 pm

OK, so we can ignore Maxnet for the moment, as just some 4-10 frames per second reach the CPU on that bridge. Triolan is a different case, there we talk about some 2000 frames per second, which is not exactly too much (the product page says the device should handle something between 29600 and 78000 frames per second when bridging in software) but also not really low.

But: those throughput measurements were done without any management interface active, and with ROS 6.x. My RB711-5HnD, which has a similar CPU (same frequency, same architecture) like the CRS112, chokes itself to death by an attempt to download a file when running ROS 7.5 (no other traffic at all):
profile.png
After less than a minute in this state, the router has rebooted.

So maybe try the same first - disconnect the ISP-related ports and try to download an upgrade file using /tool fetch url=https://download.mikrotik.com/routeros/ ... mipsbe.npk using the management bridge, watching the CPU load during the process.

What I am trying to say is that even to handle just 2000 fps may be a problem with ROS 7.5 if there is something wrong with it on the MIPSBE architecture, so preventing the traffic from those bridges from reaching the CPU may still be the solution. I cannot suggest how exactly the ACL should look like as I never had a CRS112 in my hands, but we can agree an online session.
You do not have the required permissions to view the files attached to this post.
 
User avatar
BrateloSlava
Member Candidate
Member Candidate
Topic Author
Posts: 168
Joined: Mon Aug 09, 2021 10:33 am
Location: Ukraine, Kharkiv

Re: Switch CRS112-8P-4S as a media converter

Thu Sep 22, 2022 11:43 pm

So maybe try the same first - disconnect the ISP-related ports and try to download an upgrade file using /tool fetch url=https://download.mikrotik.com/routeros/ ... mipsbe.npk using the management bridge, watching the CPU load during the process.
Some problem - free internal memory
Screenshot_Resources.png
You do not have the required permissions to view the files attached to this post.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Switch CRS112-8P-4S as a media converter

Fri Sep 23, 2022 8:54 am

Some problem - free internal memory
You can download any other file that fits by size, it need not necessarily be an upgrade package.
 
tangent
Forum Guru
Forum Guru
Posts: 1351
Joined: Thu Jul 01, 2021 3:15 pm
Contact:

Re: Switch CRS112-8P-4S as a media converter

Fri Sep 23, 2022 11:18 pm

/ip firewall connection tracking

Is that still in your config, BrateloSlava? As I tried to point out ip-thread, anything involving "/ip firewall" is going to make packets hit the CPU.

That, or it'll be ineffective because another part of the config (e.g. fast-path) causes the packets to bypass the rule.

Maybe it's time to post another sanitized RSC. All this piecemeal work trying to figure out what's current is confusing.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Switch CRS112-8P-4S as a media converter

Fri Sep 23, 2022 11:40 pm

As I tried to point out up-thread, anything involving "/ip firewall" is going to make packets hit the CPU.
Is that something new for ROS 7? I could imagine that explicit enabling of connection tracking might eventually disable L3 HW forwarding, but on ROS 6, the presence of complete IP firewall doesn't prevent L2 forwarding among switch chip ports in hardware.
 
emunt6
Frequent Visitor
Frequent Visitor
Posts: 87
Joined: Fri Feb 02, 2018 7:00 pm

Re: Switch CRS112-8P-4S as a media converter

Sat Sep 24, 2022 12:38 pm

Hi!

1., On the "CRS112-8P-4S" side:
> br-ISP1 member SFP-9, eth1
> br-ISP2 member SFP-10, eth2
> br-LAN member eth3
> Add ip address to "br-LAN" interface "172.22.0.X"
On this side, the interfaces in the different bridges - it will not forward any packet between the bridges, so don't need any firewall rules for them, its L2 only.
Now, the L2 traffic is separated, you don't any firewall rules ( L3 ), on this side.

2., On the "RB4011iGS+" side:
> br-ISP1 member eth1
> br-ISP2 member eth2
> br-LAN member eth3
> Add ip address to "br-ISP1" interface "xxxxx"
> Add ip address to "br-ISP2" interface "xxxxx"
> Add ip address to "br-LAN" interface "172.22.0.Y"
On this side, if you put every interface to the same bridge you will create L2 loop, so separation needed to prevent loop/STP block.
Now, the L2 traffic is separated, you can add your firewall rules/NAT ( L3 ) on this side.
 
User avatar
BrateloSlava
Member Candidate
Member Candidate
Topic Author
Posts: 168
Joined: Mon Aug 09, 2021 10:33 am
Location: Ukraine, Kharkiv

Re: Switch CRS112-8P-4S as a media converter

Sat Sep 24, 2022 1:02 pm

Currently, I have found CRS106-1C-5S - a certain analogue of my office switch (CRS112-8P-4S) and I am doing experiments on it. I make all settings through the console port.

Some observations:
  1. I would like to note that, immediately after a "clean" installation of ROS via Netinstall, the CPU load is very different. The system is in a hard reset state without any settings. 7.5 - 11% -20%, 6.49.6 - 1% -5%.
  2. Creating three bridges and adding ports to them doubles the CPU load. Without connecting any cables to the ports and installing optical modules.

@emunt6
I have several places where the ISP provides two IP addresses on one cable. Moreover, these addresses are from different subnets, with their own gateway values. Therefore, I organize a bridge in which I combine 3 ports. "Input" - optics, "output" - two copper cables. From the 4011 side, I "hang" the addresses provided by the provider on these two copper cables. This scheme works fine in other offices using CRS326-24G-2S+. And on this CRS112-8P-4S - problems.

4011, which are installed as routers after these switches, also have 2-3 LTE connections for organizing temporary backup communication channels. Therefore, it is "very convenient" for me that in 4011 I have each IP address on a separate interface. Easier to mark etc.
 
User avatar
BrateloSlava
Member Candidate
Member Candidate
Topic Author
Posts: 168
Joined: Mon Aug 09, 2021 10:33 am
Location: Ukraine, Kharkiv

Re: Switch CRS112-8P-4S as a media converter

Sat Sep 24, 2022 4:30 pm

Solution is wrong:
# Create new bridge

/interface bridge add name=bridge1 protocol-mode=none
           
/interface bridge port add bridge=bridge1 interface=sfp9 hw=yes
/interface bridge port add bridge=bridge1 interface=ether1 hw=yes
/interface bridge port add bridge=bridge1 interface=sfp10 hw=yes
/interface bridge port add bridge=bridge1 interface=ether2 hw=yes
/interface bridge port add bridge=bridge1 interface=ether3 hw=yes


# Maxnet (ISP1 - one IP address)
/interface ethernet switch port
set sfp9 isolation-leakage-profile-override=2
set ether1 isolation-leakage-profile-override=2
/

/interface ethernet switch port-isolation
add port-profile=2 ports=sfp9,ether1 type=dst
/


# Triolan  (ISP2 - two IP addresses)
/interface ethernet switch port
set sfp10 isolation-leakage-profile-override=0
set ether2 isolation-leakage-profile-override=1
set ether3 isolation-leakage-profile-override=1
/

/interface ethernet switch port-isolation
add port-profile=1 ports=sfp10 type=dst
/
Last edited by BrateloSlava on Fri Sep 30, 2022 7:31 pm, edited 1 time in total.
 
tangent
Forum Guru
Forum Guru
Posts: 1351
Joined: Thu Jul 01, 2021 3:15 pm
Contact:

Re: Switch CRS112-8P-4S as a media converter

Sat Sep 24, 2022 4:57 pm

As I tried to point out up-thread, anything involving "/ip firewall" is going to make packets hit the CPU.
…on ROS 6, the presence of complete IP firewall doesn't prevent L2 forwarding among switch chip ports in hardware.

There are a number of points I think we're getting separated on here.

First, I do agree that simply having a firewall doesn't push 100% of all packets to the CPU. Once the firewall makes a decision, there are a few ways it can push the decision up to the switch chip level and get out of the way. Fast-Track, L3HW, etc.

My point was, though, that in order to make that decision, a CPU somewhere needs to be involved. Whether that's the general-purpose CPU or the specialized switch SoC, though, that's the key question here.

The most directly-relevant docs don't say.

The next-most relevant I could find are ambiguous on the topic. They say that conntracking is part of prerouting, but then it goes and shows prerouting stages in both bridging (switch chip) and routing (GPCPU).

Where does the OP's conntracking go? I think only an experiment could show it, one the OP is in the best position to try.

@BrateloSlava, what happens to CPU usage if you put that one "/ip firewall connection tracking..." rule back in?
 
User avatar
BrateloSlava
Member Candidate
Member Candidate
Topic Author
Posts: 168
Joined: Mon Aug 09, 2021 10:33 am
Location: Ukraine, Kharkiv

Re: Switch CRS112-8P-4S as a media converter

Sat Sep 24, 2022 7:36 pm

@BrateloSlava, what happens to CPU usage if you put that one "/ip firewall connection tracking..." rule back in?

@tangent
  1. At the moment I have no way to check it. Only - next week.
  2. /ip firewall connection tracking set icmp-timeout=30s tcp-close-wait-timeout=1m tcp-established-timeout=1h tcp-fin-wait-timeout=2m tcp-last-ack-timeout=30s tcp-syn-received-timeout=1m tcp-syn-sent-timeout=2m tcp-time-wait-timeout=2m udp-stream-timeout=2m udp-timeout=30s

    As far as I understand, this command only changes the time intervals. It doesn't turn anything on or off. How can it affect the CPU load so drastically? It's not a command like:
    /ip firewall connection tracking set enabled=no
 
tangent
Forum Guru
Forum Guru
Posts: 1351
Joined: Thu Jul 01, 2021 3:15 pm
Contact:

Re: Switch CRS112-8P-4S as a media converter

Sat Sep 24, 2022 8:28 pm

How can it affect the CPU load so drastically?

Why do you believe an old CRS1xx era switch chip has the brains to understand L3 down to the level that it can make decisions about TCP connections?

It might, but I wouldn't bet on it.

Thus the experiment.
 
User avatar
BrateloSlava
Member Candidate
Member Candidate
Topic Author
Posts: 168
Joined: Mon Aug 09, 2021 10:33 am
Location: Ukraine, Kharkiv

Re: Switch CRS112-8P-4S as a media converter

Fri Sep 30, 2022 8:16 pm

@tangent

Careful observation of the switch showed the fallacy of the solution that I wrote above.

Everything works well, the load on the CPU is low. Until you start adding a separate port to manage the switch.
That is, as long as I controlled the switch through the console port, everything worked perfectly.

If you add one more port to this scheme to control the switch from the local network, everything collapses....

It is very similar to the fact that there is an internal collision:
- in the settings for the switching chip, I specified the profile "0" for one of the ports. This port starts acting as the switch chip's default gateway.
- on the other hand, the default gateway is also indicated in the IP settings.

It looks like this:
- while setting through the console port - CPU usage = 5-10%
- add settings
/ip address add address=172.22.0.231/24 interface=ether8 network=172.22.0.0
/ip dns set servers=172.22.0.254
/ip route add distance=1 gateway=172.22.0.254
CPU load immediately becomes 45-60%

And I tried different options:
- add this ether8 to the bridge to other ports
- describe a separate isolation profile for this ether8
- add to the bridge all ports, that are not used when transmitting from the ISP. And assign them a separate isolation profile.

Total: with such my experiments, L2 and L3 are not "friends".

And further. Changing "/ip firewall connection tracking..." had little to no effect.
 
tangent
Forum Guru
Forum Guru
Posts: 1351
Joined: Thu Jul 01, 2021 3:15 pm
Contact:

Re: Switch CRS112-8P-4S as a media converter

Fri Sep 30, 2022 9:13 pm

What does "/ip route print" say?

Will you post the new combined RSC as I asked you to above, so we don't have to piece it together in our heads?
 
User avatar
BrateloSlava
Member Candidate
Member Candidate
Topic Author
Posts: 168
Joined: Mon Aug 09, 2021 10:33 am
Location: Ukraine, Kharkiv

Re: Switch CRS112-8P-4S as a media converter

Fri Sep 30, 2022 11:27 pm

# sep/30/2022 10:06:24 by RouterOS 6.49.6
# software id = SZII-F003
#
# model = CRS112-8P-4S
/interface bridge add name=bridge1 protocol-mode=none
/interface bridge port add bridge=bridge1 interface=sfp9
/interface bridge port add bridge=bridge1 interface=ether1
/interface bridge port add bridge=bridge1 interface=sfp10
/interface bridge port add bridge=bridge1 interface=ether2
/interface bridge port add bridge=bridge1 interface=ether3
/interface ethernet switch port set 0 isolation-leakage-profile-override=2
/interface ethernet switch port set 1 isolation-leakage-profile-override=1
/interface ethernet switch port set 2 isolation-leakage-profile-override=1
/interface ethernet switch port set 8 isolation-leakage-profile-override=2
/interface ethernet switch port set 9 isolation-leakage-profile-override=0
/interface ethernet switch port-isolation add port-profile=2 ports=sfp9,ether1 type=dst
/interface ethernet switch port-isolation add port-profile=1 ports=sfp10 type=dst
/ip address add address=172.22.0.231/24 interface=ether8 network=172.22.0.0
/ip dns set servers=172.22.0.254
/ip route add check-gateway=arp distance=1 gateway=172.22.0.254
/snmp set contact=Admin enabled=yes location="MO IT" trap-version=2
/system clock set time-zone-name=Europe/Kiev
/system identity set name=rt-mo-optic
/system ntp client set enabled=yes primary-ntp=172.22.0.254
/system routerboard settings set auto-upgrade=yes
/tool bandwidth-server set enabled=no
Those lines of the configuration file that go before the line "/ip address add ,,," work fine.

If I complete all these settings completely, then the route looks like this:
[admin@rt-mo-optic] > /ip route print 
Flags: D - DYNAMIC; A - ACTIVE; c, s, y - COPY
Columns: DST-ADDRESS, GATEWAY, DISTANCE
#     DST-ADDRESS    GATEWAY       DISTANCE
0  As 0.0.0.0/0      172.22.0.254         1
  DAc 172.22.0.0/24  ether8               0
 
tangent
Forum Guru
Forum Guru
Posts: 1351
Joined: Thu Jul 01, 2021 3:15 pm
Contact:

Re: Switch CRS112-8P-4S as a media converter

Sat Oct 01, 2022 1:12 am

The dynamic route overrides the default route entirely for your LAN traffic, sending everything to ether8. Surely you want a /32 pointing to your management host here?

If you must have a /24 to allow multiple hosts behind ether8, realize that you've got a router here now, not a switch. CPU load is expected in that condition. A dumb switch isn't configured like this, which is why it can achieve wire speed.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Switch CRS112-8P-4S as a media converter

Sat Oct 01, 2022 10:16 am

@tangent, my understanding is that as long as no IP address is assigned to any Ethernet interface of the device, the CPU ignores any traffic coming from the switch chip. So it doesn't matter how "correct" or "incorrect" the IP configuration is - what matters is that it exists, because if it does, the CPU has to deal with the received frames (if only to find out that they are not interesting) and that causes the load, hich for some reason is higher in ROS 7 than in ROS 6 for the same amount of frames. @BrateloSlava does not use the device as a router - he's got only a single IP address attached to a single interface (for management purposes), so no routing is even possible.

Which only confirms what I keep saying from the beginng - using the switch chip ACL to prevent the switch chip from sending any frames from the two bridges to the CPU-facing port would prevent them from loading the CPU. The question is whether it is worth the effort, i.e. whether management/monitoring of the device via IP is necessary or whether one-time configuration using the serial console is sufficient.
 
User avatar
BrateloSlava
Member Candidate
Member Candidate
Topic Author
Posts: 168
Joined: Mon Aug 09, 2021 10:33 am
Location: Ukraine, Kharkiv

Re: Switch CRS112-8P-4S as a media converter

Sat Oct 01, 2022 1:37 pm

The question is whether it is worth the effort, i.e. whether management/monitoring of the device via IP is necessary or whether one-time configuration using the serial console is sufficient.
@sindy
It seems, that the answer to this question becomes fundamental in my problem. :)

I'll do some more experiments to find out how "good" the switching chip in this device is when working with VLANs. After that, most likely, I will return to the configuration through the console port without assigning an IP address for management.
 
User avatar
BrateloSlava
Member Candidate
Member Candidate
Topic Author
Posts: 168
Joined: Mon Aug 09, 2021 10:33 am
Location: Ukraine, Kharkiv

Re: Switch CRS112-8P-4S as a media converter  [SOLVED]

Thu Oct 13, 2022 11:56 pm

The problem has a completely normal solution. The load on the central processor did not exceed 25% with uTP traffic of 400 Mbps in one direction. Higher speeds could not be tested due to the limitations of the routers that I used.

To test, I assembled the following kit:
- Mikrotik hAP ac2 router (mt-isp-router) was used as "ISP". To check the traffic flow, it was connected via WiFi to my home network.
- the Mikrotik CRS106-1C-5S switch (mt-isp-optic-switch) was connected with a copper cable to the router (mt-isp-router) and was used as an "ISP" end device to which clients were to connect
- the Mikrotik CRS106-1C-5S switch (mt-client-optic-switch) was connected with two optical cables to the "ISP" switch (mt-isp-optic-switch) to create a situation where the client has two independent connections to two ISPs
- a Mikrotik RB751G-2HnD router (mt-client-router) was used as a client router and was connected with two copper cables to an optical switch (mt-client-optic-switch)

SFP-2-RG45.png

Step4Net SFPd-03-1550-WDM-LC and Step4Net SFPd-03-1310-WDM-LC were used as optical modules.
Considering that the Mikrotik CRS106-1C-5S switch has only one port for connecting via a copper cable, I used Mikrotik S-RJ01.

The task of minimizing the load on the central processor of the client's switch was solved when connecting to several providers via several optical cables.
The client router is connected with several copper cables.
The client must be able to monitor their switch and router from their local network.

Below are the settings I made for each device. For both routers, these settings are not complete and are used only to show the principle of setting.
The Mikrotik CRS106-1C-5S switching chip showed quite "normal behavior" for a device in this price range.

--
ISP router (mt-isp-router)
--

# oct/13/2022 18:38:43 by RouterOS 7.5
# software id = QQ55-906W
#
# model = RBD52G-5HacD2HnD
# serial number = 

### Main settings
/interface list add name=LAN
/interface list add name=WAN

/interface bridge add ingress-filtering=no name=bridge1 vlan-filtering=yes

/interface vlan add interface=bridge1 name=vlan-ISP-LAN-100 vlan-id=100
/interface vlan add interface=bridge1 name=vlan-client-30 vlan-id=30
/interface vlan add interface=bridge1 name=vlan-client-31 vlan-id=31

/interface bridge port add bridge=bridge1 interface=ether1 pvid=100
/interface bridge port add bridge=bridge1 interface=ether2 pvid=100
/interface bridge port add bridge=bridge1 interface=ether3 pvid=100
/interface bridge port add bridge=bridge1 interface=ether4 pvid=100
/interface bridge port add bridge=bridge1 interface=ether5

/interface bridge vlan add bridge=bridge1 tagged=bridge1,ether5 untagged=ether1,ether2,ether3,ether4 vlan-ids=100
/interface bridge vlan add bridge=bridge1 tagged=bridge1,ether5 vlan-ids=30
/interface bridge vlan add bridge=bridge1 tagged=bridge1,ether5 vlan-ids=31

/interface list member add interface=wlan2 list=WAN
/interface list member add interface=vlan-ISP-LAN-100 list=LAN
/ip neighbor discovery-settings set discover-interface-list=LAN

### Other settings
/interface wireless security-profiles set [ find default=yes ] supplicant-identity=MikroTik
/interface wireless security-profiles add authentication-types=wpa2-psk mode=dynamic-keys name=profile-Test-ISP supplicant-identity="" wpa2-pre-shared-key=SuperSecretPassword
/interface wireless set [ find default-name=wlan2 ] adaptive-noise-immunity=ap-and-client-mode band=5ghz-a/n/ac channel-width=20/40/80mhz-Ceee comment=WAN country=ukraine disabled=no distance=indoors hw-protection-mode=rts-cts security-profile=profile-Test-ISP ssid=APTestISP wireless-protocol=802.11
/interface wireless nstreme set wlan2 comment=WAN
/interface wireless manual-tx-power-table set wlan2 comment=WAN
/ip pool add name=dhcp_pool_LAN_ISP ranges=10.10.100.1-10.10.100.253
/ip pool add name=dhcp_pool-client-30 ranges=10.10.30.1-10.10.30.253
/ip pool add name=dhcp_pool-client-31 ranges=10.10.31.1-10.10.31.253
/ip dhcp-server add address-pool=dhcp_pool_LAN_ISP interface=vlan-ISP-LAN-100 name=dhcp-LAN-ISP
/ip dhcp-server add address-pool=dhcp_pool-client-30 interface=vlan-client-30 name=dhcp-client-30
/ip dhcp-server add address-pool=dhcp_pool-client-31 interface=vlan-client-31 name=dhcp-client-31
/ipv6 settings set disable-ipv6=yes
/ip address add address=10.10.30.254/24 interface=vlan-client-30 network=10.10.30.0
/ip address add address=10.10.31.254/24 interface=vlan-client-31 network=10.10.31.0
/ip address add address=10.10.100.254/24 interface=vlan-ISP-LAN-100 network=10.10.100.0
/ip dhcp-client add interface=wlan2
/ip dhcp-server network add address=10.10.30.0/24 dns-server=1.1.1.1,8.8.8.8 gateway=10.10.30.254
/ip dhcp-server network add address=10.10.31.0/24 dns-server=1.0.0.1,8.8.4.4 gateway=10.10.31.254
/ip dhcp-server network add address=10.10.100.0/24 dns-server=10.10.100.254 gateway=10.10.100.254
/ip dns set allow-remote-requests=yes cache-max-ttl=10m
/ip firewall nat add action=masquerade chain=srcnat comment="Access LAN to Internet" ipsec-policy=out,none out-interface-list=WAN
/ip firewall service-port set ftp disabled=yes
/ip firewall service-port set tftp disabled=yes
/ip firewall service-port set irc disabled=yes
/ip firewall service-port set h323 disabled=yes
/ip firewall service-port set sip disabled=yes
/ip firewall service-port set pptp disabled=yes
/ip service set www disabled=yes
/ip service set api disabled=yes
/ip service set api-ssl disabled=yes
/system clock set time-zone-name=Europe/Kiev
/system identity set name=MT-ISP-Router
/system ntp client set enabled=yes
/system routerboard settings set auto-upgrade=yes

--
ISP switch (mt-isp-optic-switch)
--

# oct/13/2022 18:41:13 by RouterOS 7.5
# software id = WR66-JAP8
#
# model = CRS106-1C-5S
# serial number = 


### Main settings
/interface list add name=LAN

/interface bridge add name=bridge1 protocol-mode=none

/interface vlan add interface=bridge1 name=ISP-LAN-VLAN-100 vlan-id=100

/interface bridge port add bridge=bridge1 interface=sfp1
/interface bridge port add bridge=bridge1 interface=sfp2
/interface bridge port add bridge=bridge1 interface=sfp3
/interface bridge port add bridge=bridge1 interface=sfp4
/interface bridge port add bridge=bridge1 interface=sfp5
/interface bridge port add bridge=bridge1 interface=combo1

/interface ethernet switch egress-vlan-tag add tagged-ports=combo1 vlan-id=30
/interface ethernet switch egress-vlan-tag add tagged-ports=combo1 vlan-id=31

/interface ethernet switch ingress-vlan-translation add customer-vid=0 new-customer-vid=30 ports=sfp1
/interface ethernet switch ingress-vlan-translation add customer-vid=0 new-customer-vid=31 ports=sfp2

/interface ethernet switch vlan add ports=sfp1,combo1 vlan-id=30
/interface ethernet switch vlan add ports=combo1,sfp2 vlan-id=31

/interface ethernet switch vlan add ports=switch1-cpu,combo1 vlan-id=100


/ip dhcp-client add interface=ISP-LAN-VLAN-100

/interface list member add interface=ISP-LAN-VLAN-100 list=LAN
/ip neighbor discovery-settings set discover-interface-list=LAN

### Other settings
/ipv6 settings set disable-ipv6=yes
/ip service set www disabled=yes
/ip service set api disabled=yes
/ip service set api-ssl disabled=yes
/system clock set time-zone-name=Europe/Kiev
/system identity set name=MT-ISP-Optic
/system ntp client set enabled=yes
/system routerboard settings set auto-upgrade=yes
/tool bandwidth-server set enabled=no

--
Client switch (mt-client-optic-switch)
--

# oct/13/2022 18:35:45 by RouterOS 7.5
# software id = YWA1-ED7I
#
# model = CRS106-1C-5S
# serial number =

### Main settings
/interface list add name=LAN

/interface bridge add name=bridge1 protocol-mode=none

/interface vlan add interface=bridge1 name=Client-LAN-VLAN-101 vlan-id=101

/interface ethernet switch trunk add member-ports=sfp5,combo1 name=trunk-for-router

/interface bridge port add bridge=bridge1 interface=sfp1
/interface bridge port add bridge=bridge1 interface=sfp2
/interface bridge port add bridge=bridge1 interface=sfp3
/interface bridge port add bridge=bridge1 interface=sfp4
/interface bridge port add bridge=bridge1 interface=sfp5
/interface bridge port add bridge=bridge1 interface=combo1


/interface ethernet switch egress-vlan-tag add tagged-ports=trunk-for-router vlan-id=130
/interface ethernet switch egress-vlan-tag add tagged-ports=trunk-for-router vlan-id=131

/interface ethernet switch ingress-vlan-translation add customer-vid=0 new-customer-vid=130 ports=sfp1
/interface ethernet switch ingress-vlan-translation add customer-vid=0 new-customer-vid=131 ports=sfp2

/interface ethernet switch vlan add ports=trunk-for-router,sfp1 vlan-id=130
/interface ethernet switch vlan add ports=trunk-for-router,sfp2 vlan-id=131

/interface ethernet switch vlan add ports=trunk-for-router,switch1-cpu vlan-id=101


/ip dhcp-client add interface=Client-LAN-VLAN-101

/interface list member add interface=Client-LAN-VLAN-101 list=LAN
/ip neighbor discovery-settings set discover-interface-list=LAN

### Other settings
/ipv6 settings set disable-ipv6=yes
/ip service set www disabled=yes
/ip service set api disabled=yes
/ip service set api-ssl disabled=yes
/system clock set time-zone-name=Europe/Kiev
/system identity set name=MT-Client-Optic
/system ntp client set enabled=yes
/system routerboard settings set auto-upgrade=yes
/tool bandwidth-server set enabled=no

--
Client router (mt-client-router)
--

# oct/13/2022 18:31:53 by RouterOS 7.5
# software id = Z3DJ-FCXD
#
# model = 751G-2HnD
# serial number = 

### Main settings
/interface list add name=LAN
/interface list add name=WAN

/interface bridge add name=bridge1

/interface bonding add forced-mac-address=XX:XX:XX:XX:XX:XX mode=balance-xor name=bonding-2-switch slaves=ether1,ether2 transmit-hash-policy=layer-2-and-3

/interface vlan add interface=bonding-2-switch name=IP-ISP1-VLAN-130 vlan-id=130
/interface vlan add interface=bonding-2-switch name=IP-ISP2-VLAN-131 vlan-id=131
/interface vlan add interface=bonding-2-switch name=vlan-LAN-101 vlan-id=101

/interface bridge port add bridge=bridge1 interface=ether5
/interface bridge port add bridge=bridge1 interface=ether4
/interface bridge port add bridge=bridge1 interface=ether3
/interface bridge port add bridge=bridge1 interface=vlan-LAN-101

/interface list member add interface=IP-ISP1-VLAN-130 list=WAN
/interface list member add interface=IP-ISP2-VLAN-131 list=WAN
/interface list member add interface=bridge1 list=LAN

/ip neighbor discovery-settings set discover-interface-list=LAN
/ip dhcp-client add interface=IP-ISP1-VLAN-130
/ip dhcp-client add interface=IP-ISP2-VLAN-131

### Other settings
/ipv6 settings set disable-ipv6=yes
/ip pool add name=dhcp_pool-client-LAN ranges=192.168.190.1-192.168.190.253
/ip dhcp-server add address-pool=dhcp_pool-client-LAN interface=bridge1 name=dhcp-Client-LAN
/ip address add address=192.168.190.254/24 interface=bridge1 network=192.168.190.0
/ip dhcp-server network add address=192.168.190.0/24 dns-server=1.1.1.1,8.8.8.8,1.0.0.1,8.8.4.4 gateway=192.168.190.254
/ip dns set allow-remote-requests=yes cache-max-ttl=5m
/ip firewall nat add action=masquerade chain=srcnat comment="Access LAN to Internet" ipsec-policy=out,none out-interface-list=WAN
/system clock set time-zone-name=Europe/Kiev
/system identity set name=MT-Client-Router
/system ntp client set enabled=yes
/system routerboard settings set auto-upgrade=yes
You do not have the required permissions to view the files attached to this post.

Who is online

Users browsing this forum: AndyGs, hatred, koer, xristostsilis and 103 guests