We have this scenario, (click on image to view in wide)
we have connected the switch and the routers with the with S+DA0003 - SFP DACs (https://mikrotik.com/product/SplusDA0003), the switch only have 4 bridges with 3 interface in each one, all is configured by default, no networks or firewall or nating rules and basically is working a a simple switch with this 4 bridges, nothing more.
all ISP have 1Gbps symmetric links.
when we make a speed test of this setup, the D/U speed is not over than 300Mbps, ever. Now, if we connect directly the ONT of the ISP to the routers the speed of D/U is 1Gbps, ever.
this is a hardware limitation or a misconfiguration? we can not understand why the performance is too poor if is working like a switch/bridge.
You forgot to make the file with the drawing accessible for everyone.
Post your configuration (see the hint in my automatic signature) - maybe hardware “acceleration” (actually, direct forwarding done by the switch chip) is disabled on your bridges?
Okay. The Port switching chapter of Manual:CRS3xx series switches says:
Warning: Currently it is possible to create only one bridge with hardware offloading on CRS3xx series devices. Use the hw parameter to select which bridge will use hardware offloading.
As you have defined 4 independent bridges, the chance that you hit the one which the RouterOS has chosen to benefit from hardware offloading is 25 %.
So to have full wirespeed, you have to use a single bridge where the individual ports will be configured as access ports to different VLANs. You’ll probably want to use S-VLAN tags, and may need to use /interface ethernet switch rule configuration subtree - there is an ongoing integration of VLAN handling by bridge and switch chip, so start from configuring /interface bridge port and /interface bridge vlan for use with vlan-filtering=yes on the single bridge (with ether-type set to the default 0x8100 if you want to use C-VLANs or to 0x88a8 if you want to use S-VLANs); if it doesn’t work as expected, start reading the /interface ethernet switch rule chapter.
thanks for your answer, i’ve tested the second option than you mention “Or take the simpler path with no VLANs at all, and use the switch chip configuration to set up port isolation on a single bridge without VLANs.” and the speed was improved, but i’ve detected packet losted doing pings, i’ve tested first only with 2 ISP (6 interfaces in one bridge with 2 groups)
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=52 time=13.0 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=52 time=12.8 ms
64 bytes from 8.8.8.8: icmp_seq=4 ttl=52 time=12.8 ms
64 bytes from 8.8.8.8: icmp_seq=7 ttl=52 time=12.5 ms
64 bytes from 8.8.8.8: icmp_seq=8 ttl=52 time=12.6 ms
64 bytes from 8.8.8.8: icmp_seq=12 ttl=52 time=12.2 ms
64 bytes from 8.8.8.8: icmp_seq=14 ttl=52 time=13.0 ms
64 bytes from 8.8.8.8: icmp_seq=15 ttl=52 time=12.7 ms
64 bytes from 8.8.8.8: icmp_seq=16 ttl=52 time=13.3 ms
64 bytes from 8.8.8.8: icmp_seq=17 ttl=52 time=12.7 ms
64 bytes from 8.8.8.8: icmp_seq=18 ttl=52 time=13.1 ms
^C
--- 8.8.8.8 ping statistics ---
18 packets transmitted, 11 received, 38% packet loss, time 17123ms
rtt min/avg/max/mdev = 12.246/12.849/13.330/0.325 ms
that happens with that 2 ISP, intermittently you know what can be?
I don’t but given the round-trip time and the speed, I’d say there are quite some hops between your hardware and the nearest instance of Google DNS. There is not much reason why a mere L2 network with hardware forwarding should have such a high drop rate, so if I was to debug it, I’d connect one PC instead of ISP 1 and try pinging it from the one from which you pinged 8.8.8.8. If you can see the same drop rate, something is wrong with the switches themselves or, much more likely, with the interconnections between them (dirty or ill-crimped cables in cause if copper connections, dirty connectors or too bent cables in case of optical connections). If there are no drops, the switches and their interconnections are OK and the issue is somewhere between the uplink port of the switch and the Google DNS instance, so in the next step I’d connect the test PC directly to the ISP path using the same cable previously used to connect the switch. If the above is impossible due to different interface types, a microscope, and optical power meter and cleaning wipes in case of optical connections, and crimping tool and a bag of new 8p8c connectors or just a bag of new patchcords are the next tools you need.
Error counters on switch interfaces can tell you where a cabling issue is, unless the packets get malformed on the uplink between the switch and the ISP as you can’t see ISP device’s counters.
Hi sindy, thanks for your answer, I’ve can’t test before for other thing than i’m working.
the ONTs in fact are SFP GPON ONU Sticks with the ISP fibre attached directly and the interconnections between the routers and the switch are mikrotik DACs fiber cables S+DA0003 as i’ve mentioned before.
to discard problems with the uplinks and know if the problems are related to the switch, i’ve tested pings and i’ve generated traffic in the next scheme
like this no packages lost and any performance issue.
when i’ve replace the PC2 with the switch feel the same issue mentioned before (packets lost, slow performance) PC1 ------utp cat7-----------> ROUTER ----------DAC SFP cable -----------> SWITCH + SFP GPON ONU Stick ----------- ISP Fibre ---------------> OLT (ISP side)
i’ve reset the configurations in the switch again and i’ve configured following your second recommendation (without VLANs) the problem persist.
I feel than something is missing or bad configured on the switch, this is very weird.
all the problem was solved reconfiguring all again in the bridge switch
Current config:
1 - only one bridge with all ports inside and Hardware of OffLoad enabled
2 - Isolate the ports than we want E.g:
port1 + port2 + port9 = line ISP1 and trunk for each router
port3 + port4 + port11 = line ISP2 and trunk for each router
port5 + port6 + port13 = line ISP3 and incoming for each router
etc…
