Packages arriving jerky

Daniel44 · Mon Sep 05, 2022 9:24 am

Hello,

I have a CRS-106-1c-6s configured as follows:

[admin@switch_b-test3] > /export compact
# jun/01/1971 18:43:01 by RouterOS 6.48.1
# software id = LGUY-0R4B
#
# model = CRS106-1C-5S
# serial number = 6DE007126090
/interface bridge
add name=bridge protocol-mode=none vlan-filtering=yes
/interface vlan
add interface=bridge name=vlan1337 vlan-id=1337
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/interface bridge port
add bridge=bridge comment=Haus frame-types=admit-only-untagged-and-priority-tagged interface=combo1
add bridge=bridge comment=Zentrale interface=sfp1
add bridge=bridge comment=defconf frame-types=admit-only-vlan-tagged interface=sfp3
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=sfp4 pvid=1337
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=sfp5 pvid=1337
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=sfp2 pvid=1337
/interface bridge vlan
add bridge=bridge tagged=sfp1 untagged=sfp3,sfp4,sfp5 vlan-ids=1337
/ip address
add address=192.168.88.3/24 comment=defconf interface=bridge network=192.168.88.0
add address=194.120.111.252/22 comment="Adresse  vlan internal" interface=vlan1337 network=194.120.108.0
/system identity
set name=switch_b-test3

On the VLAN there is hardware which is manufactured by us. On sfp1 is a machine with linux, which has a configured VLAN interface.
On SFP2, 4 and 5 is our Hardware.
We have a program, which sends a large amount of packages to that hardware and logs for transfer errors.
When I send a lot of packages to a device, they arrive in a jerky manner.
That doesn't happen with a old Netgear GS108T. With this switch it works as expected.

Any Idea what can be the cause of this or what I can do to find the cause?

Regards

Daniel

mkx · Mon Sep 05, 2022 3:01 pm

CRS1xx can't offload bridge operations to hardware, which means all manipulations are done by its general-purpose CPU. In order for switch chip to do the processing (at wirespeed), you have to reconfigure your CRS106 to use switch chip which is configured under /interface ethernet switch and subtree ... according to manual examples.

Daniel44 · Mon Sep 05, 2022 3:50 pm

You mean the Hardware-offload?

[admin@switch_b-test3] /interface/bridge/port> print
Flags: I - INACTIVE
Columns: INTERFACE, BRIDGE, HW, PVID, PRIORITY, PATH-COST, INTERNAL-PATH-COST, HORIZON
#   INTERFACE  BRIDGE  HW   PVID  PRIORITY  PATH-COST  INTERNAL-PATH-COST  HORIZON
;;; Haus
0   combo1     bridge  yes     1  0x80             10                  10  none   
;;; Zentrale
1   sfp1       bridge  yes     1  0x80             10                  10  none   
;;; defconf
2   sfp4       bridge  yes  1337  0x80             10                  10  none   
;;; defconf
3   sfp5       bridge  yes  1337  0x80             10                  10  none   
4   sfp2       bridge  yes  1337  0x80             10                  10  none   
5 I sfp3       bridge  yes     1  0x80             10                  10  none

mkx · Mon Sep 05, 2022 7:39 pm

Yes, I mean HW offload. What you see in column #5 is configured value. What you don't see, are 'H' in column #2, showing that hardware offload is indeed active. Something like this:

/interface/bridge/port print
Flags: I - INACTIVE; H - HW-OFFLOAD
Columns: INTERFACE, BRIDGE, HW, PVID, PRIORITY, PATH-COST, INTERNAL-PATH-COST, HORIZON
#    INTERFACE      BRIDGE  HW   PVID  PRIORITY  PATH-COST  INTERNAL-PATH-COST  HORIZON
0  H ether1-trunk   bridge  yes     1  0x80             10                  10  none
1 IH ether2         bridge  yes     1  0x80             10                  10  none
2 IH ether3         bridge  yes     1  0x80             10                  10  none
3 IH ether4         bridge  yes     1  0x80             10                  10  none
4  H ether5         bridge  yes     1  0x80             10                  10  none
5    wifi-42        bridge          1  0x80             10                  10  none
6 I  ILoveMikrotik  bridge          1  0x80             10                  10  none

The last two are wireless interfaces, not fit for HW offload at all. The rest are offloaded even though the device is configured for VLANs:

/interface bridge
add admin-mac=E6:8D:8C:C6:11:C4 auto-mac=no name=bridge
/interface bridge port
add bridge=bridge interface=ether1-trunk
add bridge=bridge interface=ether2
add bridge=bridge interface=ether3
add bridge=bridge interface=ether4
add bridge=bridge interface=ether5
add bridge=bridge interface=wifi-42
add bridge=bridge interface=ILoveMikrotik
/interface ethernet
set [ find default-name=ether1 ] name=ether1-trunk
/interface ethernet switch port
set ether1-trunk vlan-mode=secure
set ether2 default-vlan-id=40 vlan-header=always-strip vlan-mode=secure
set ether3 default-vlan-id=42 vlan-header=always-strip vlan-mode=secure
set ether4 default-vlan-id=42 vlan-header=always-strip vlan-mode=secure
set ether5 vlan-mode=secure
set switch1-cpu vlan-mode=secure
/interface ethernet switch vlan
add independent-learning=yes ports=switch1-cpu,ether1-trunk,ether5 switch=switch1 vlan-id=99
add independent-learning=yes ports=switch1-cpu,ether1-trunk,ether3,ether4,ether5 switch=switch1 vlan-id=42
add independent-learning=yes ports=switch1-cpu,ether1-trunk switch=switch1 vlan-id=41
add independent-learning=yes ports=ether1-trunk,ether2 switch=switch1 vlan-id=3999
add independent-learning=yes ports=ether1-trunk,ether2 switch=switch1 vlan-id=40
add independent-learning=yes ports=ether1-trunk,ether5 switch=switch1 vlan-id=2

(note that I replaced index numbers with interface names in /interface ethernet switch port above for better understanding of settings. The export commad has idnex numbers there).

Mind that device above is RB951G and has different configuration options (and syntax) under /interface ethernet, so you have to refer to manual page I linked in my previous post.

Daniel44 · Tue Sep 06, 2022 8:42 am

Ok,

so hardware-offload is active on all of my ports. (see posting.php?mode=reply&t=188954&sid=005 ... 5#pr955504)
Any ideas how to find out, whats going on here?

Regards

Daniel

elbob2002 · Tue Sep 06, 2022 9:30 am

As explained already the CRS-1XX Series don't support bridge hardware offloading.

From the documentation:

For example, you use this configuration on a CRS1xx/CRS2xx series device and you started to notice that the CPU usage is very high and when running a performance test to check the network's throughput you notice that the total throughput is only a fraction of the wire-speed performance that it should easily reach. The cause of the problem is that not all devices support bridge VLAN filtering on a hardware level. All devices are able to be configured with bridge VLAN filtering, but only a few of them will be able to offload the traffic to the switch chip. If an improper configuration method is used on a device with a built-in switch chip, then the CPU will be used to forward the traffic.

Create a single default bridge and add all your interfaces to it. Now use the Switch Menu to configure your VLANs as per the link @mkx already posted.

Daniel44 · Wed Sep 07, 2022 11:35 am

Ok, thats too much for me.
Seems, that the RouterOS isn't quite handy but very flexible. Too flexible for me

Daniel44 · Fri Sep 09, 2022 11:45 am

If a wan't to have a port with a tagged VLAN 1337 but also as a untagged Port for the default VLAN (0 I think)....how do I have to configure this? Do I have to create a VLAN 0 in the switch section?
And How do I have to set the ingres and egress vlan tags?

Regards

Daniel

mkx · Fri Sep 09, 2022 3:53 pm

The untagged VLAN is tretated slightly differently by different switch chips. I suggest you to go "all-tagged". Which means you have another VLAN used internally and have ports, that should be untagged externally, configured as access ports to that VLAN. This can also serve as future proof config in case you decide to connect your untagged VLAN via some trunk connections to another switch.

When selecting VLAN ID for "untagged VLAN": 0 is illegal value (legal VLAN ID numbers are from 1 to 4094), you could go for 1 which is used implicitly in ROS (and by some other vendors) but due to implicit config all over the place it may give some problems. So I suggest you to avoid ID 1 (mind that there's nothing wrong with using VID 1 ... if you manage to change config everywhere that's needed).

Then you'd go something like this:

/interface bridge
add name=bridge  # normally there will only be one bridge per switch ... functions of second and further bridges can not be offloaded to hardware

/interface bridge port
add bridge=bridge interface=ether6 hw=yes # add all the ports that are used to connect same (V)LANs, normally this means all of ports

/interface ethernet switch ingress-vlan-translation  # this section is about untagged ports (or hybrid ports for untagged part)
add ports=ether6 customer-vid=0 new-customer-vid=666   # or whatever ID you select for "untagged VLAN" and whatever ether port you want to be a hybrid port

/interface ethernet switch egress-vlan-tag  # this section is about tagged ports (or tagged part of hybrid ports)
add tagged-ports=ether6 vlan-id=1337

/interface ethernet switch vlan  # this section is about all VLANs, both tagged and untagged
add ports=ether6 vlan-id=1337
add ports=ether6 vlan-id=666

Repeat hte configuration regarding ether6 for all other ports according to wanted config.

If you want switch to interact with untagged VLAN (e.g. if it's your management vlan at the same time), then you have to add some config to make bridge interface part of that VLAN. I suggest you to make it tagged member of said VLAN:

# this code block replaces some of config and adds some to the code block above
/interface ethernet switch egress-vlan-tag
add tagged-ports=switch1-cpu vlan-id=666

/interface ethernet switch vlan
set [ find vlan-id=666 ] ports=ether6,switch1-cpu  # make sure the port list is complete ... you can not "add" a port to already existing list

/interface vlan
add interface=bridge name=vlan666 vlan-id=666

Then you normally use interface vlan666 as interface having switch's IP address assigned etc. (instead of bridge if it was simple switch without VLANs or a physical interface (e.g. ether6) if the physical interface was exclusively dedicated to management of device).

You would similarly go if management was done over tagged VLAN ...

Daniel44 · Mon Sep 12, 2022 9:22 am

If I understand it correctly, the management interface is configured via the switch1-cpu?

Oh, and there seems to be already a "default" Vlan in the switch section....VLAN-ID 4095

What is strange in my opninion: I can ping Vlan-Host2 from VLAN-Host1 without configured VLANs, simply factory defaults....

mkx · Mon Sep 12, 2022 5:58 pm

I'm not sure if config, posted in original post, is still vaild. So I can not commento n your observation. But generally ROS device will try to route between all of interfaces with IP address configured unless there are firewall rules blocking such communication. In configuration from original post tehre are two interfaces with IP addresses set: bridge interfacewith IP address 192.168.88.3/24 and vlan1337 interface with IP address 194.120.11.252/22 and no firewall rules that might block communication between these two interfaces.

When CRS is used as a swtich with multiple VLANs passing between ports, switch does not need /interface vlan interface for switching functionality. A switch only needs single interface with IP address set and that should be used only for management of the switch.

If you want to get some more concrete help with your configuration, post actual configuration (again) - and we'll tear it apart

Daniel44 · Tue Sep 13, 2022 8:50 am

I have no "current config" so far.´The bare board of the switch is build into our hardware. And to have a chance to configure ist, we only put a "default config" on it, which made it possible to talk to the board and protect it with a password.

The config I posted first was our "first try" (with some help from here) to get a working VLAN solution. But with this config we have the problem of the stucking when transfering a large amount of packages.

What we need:

Combo1: the "normal" network packages (I think its VLAN-ID 0). Via this device, the Switch is reachable from the lan.

SFP1: "normal" network traffic (VLAN-ID 0) PLUS tagged VLAN-Packages with VLAN-ID 1337
SFP3: Untagged packages of VLAN-ID 1337, incomming packages must be tagged with VLAN 1337
SFP4: Untagged packages of VLAN-ID 1337, incomming packages must be tagged with VLAN 1337
SFP5: Untagged packages of VLAN-ID 1337, incomming packages must be tagged with VLAN 1337
Management Interface should be available via the VLAN 1337, IP 192.168.0.239 (old default IP of the netgear switches we build in before)

Thats all I think.
With the old netgear GS108T it was no problem, simply some clicks. Now to get the same result with the Mirkotik it costs me weeks. So I'm a bit frustrated

Best regards

DAniel

mkx · Tue Sep 13, 2022 9:06 am

Perhaps somebody else will chime in. I don't want to give you full config for your device because

I don't own a CRS1xx so I can't be 100% sure my config would be correct
I'm willing to aide with learning how to use Mikrotik devices but I'm not a consultant (even less a paid one)

So if you're unable to work from the "first try" config you posted and through suggestions I already provided (and some other might add some suggestions), then you probably have to get some consultant to do the job for you. Or get switch(es) from some other vendor which have easier management UI.

Daniel44 · Tue Sep 13, 2022 10:53 am

The Schema I gave you is not what we use outside.
Its a setup here in my office to play and to learn with.
The final config I will do by myself. I only need a hook to hang on...

We delivered already switches with a config which works basically, but not at high loads.

Daniel44 · Tue Sep 13, 2022 11:14 am

Thats my config so far, seems to work as I expected. Didn't test it under load untill now.
The only thing I miss atm. is how to get the management interface available via the vlan 1337

# jan/02/1970 00:50:38 by RouterOS 7.5
# software id = LGUY-0R4B
#
# model = CRS106-1C-5S
# serial number = 6DE007126090
/interface bridge
add admin-mac=64:D1:54:80:A4:1C auto-mac=no comment=defconf name=bridge
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip hotspot profile
set [ find default=yes ] html-directory=hotspot
/port
set 0 baud-rate=115200 name=serial0
/interface bridge port
add bridge=bridge comment=defconf interface=combo1
add bridge=bridge comment=defconf interface=sfp1
add bridge=bridge comment=defconf interface=sfp2
add bridge=bridge comment=defconf interface=sfp3
add bridge=bridge comment=defconf interface=sfp4
add bridge=bridge comment=defconf interface=sfp5
/interface ethernet switch egress-vlan-tag
add tagged-ports=sfp1,sfp2 vlan-id=1337
/interface ethernet switch ingress-vlan-translation
add customer-vid=0 new-customer-vid=1337 ports=sfp3,sfp4,sfp5
/interface ethernet switch vlan
add ports=switch1-cpu,sfp1,sfp2,sfp3,sfp4,sfp5 vlan-id=1337
/ip address
add address=192.168.88.1/24 comment=defconf interface=bridge network=192.168.88.0
add address=194.120.111.250 interface=bridge network=255.255.252.0

Edit: Ok, tested it, still slow and jerky :/

mkx · Tue Sep 13, 2022 4:57 pm

For the management access:

/interface vlan
add interface=bridge name=vlan1337 vlan-id=1337

/ip address
add address=x.y.z.w/24 interface=vlan1337

Substitute x.y.z.w with actual IP address you want to use when managing switch. If it's one of already assigned IP addresses, then you'll have to deassign that address first (and make sure you're doing it while connecting to switch using the other IP address ... or by connecting via MAC using winbox).

After you verify you can actually connect to the newly set-up management IP address, remove all other IP addresses. If at this point communication between LAN devices ceases to flow, then CRS was actually routing traffic and CRS1xx is a very low-capacity router (if configured as such).

Other options you can try:

remove combo1 port from bridge. If you need this port as local management port, configure IP address directly on port.
In theory combo1 is part of switch chip and should work just like the other ports, but since it's combo, who knows.
what kind of SFP modules are you using, optical ones? If so, take extra care to have connectors clean (both cable side as well as SFP side).
Dirty fibre connections can cause packet drops which in turn cause all sorts of problems with data transfer (either excessive retransmissions which translates to jerky transfer speeds or even dropped contents in case when retrasmissions are not part of transport protocol - multicasts and UDP don't offer retrasmissions on transport layer).
If you're using RJ45 SFP modules, check if they are on compatibility list. MT devices can be quite picky about which SFP modules ... and that's true both for RJ45 modules and optical modules.

Packages arriving jerky

Packages arriving jerky

Re: Packages arriving jerky

Re: Packages arriving jerky

Re: Packages arriving jerky

Re: Packages arriving jerky

Re: Packages arriving jerky

Re: Packages arriving jerky

Re: Packages arriving jerky

Re: Packages arriving jerky

Re: Packages arriving jerky

Re: Packages arriving jerky

Re: Packages arriving jerky

Re: Packages arriving jerky

Re: Packages arriving jerky

Re: Packages arriving jerky

Re: Packages arriving jerky

Who is online