Poor vlan performance on set of Mikrotik - tagged vlan's

RouterOS General
1

Hi gurus,

I have three MikroTik devices:

a) Mikrotik CCR-1009-8G-1S-1S+ ROS 6.19
b) Mikrotik RB2011UAS-2HnD with SFP module ROS 6.20
c) Mikrotik CRS-125-24G-1S ROS 6.19

They are connected A to B and A to C with leased fiber connection with tagged vlans - one per location. On site C is installed mediaconverter and it’s connected to ether24, On site A those two vlans are going into one ethernet cable and goes to port ether2.
I’m using routing to connect this sites - between A and B I have adresses 172.18.0.0/30, between A and C 172.18.0.4/30.

Im having issues with performance of this setup - beeing behind each router I cant achieve more then 16mbps.
Strange thing is, that btests are working very well - I achieve 100% of speed on each site, both on UDP and TCP tests

On B and C sites NAT is disabled and there are few (just a bunch of) firewall rules, but after disabling them I cant see any difference.

I think, that problem might be in this VLAN setup on CCR or maybe different MTUs?
Between devices are some other switches, because it’s not dedicated line, but just leased transmission.

I tried:

  • setting down MTU (to 1400) - without result
  • changing default queue to ethernet-default on all three interfaces
/interface ethernet
set [ find default-name=ether1 ] name=ether1-WAN-3s
set [ find default-name=ether2 ] full-duplex=no name=ether2 speed=1Gbps
set [ find default-name=ether3 ] name=ether3-virtual
set [ find default-name=ether4 ] name=ether4-virtual2
set [ find default-name=ether5 ] name=ether5-nas
set [ find default-name=ether6 ] name=ether6-virtual4
set [ find default-name=ether7 ] name=ether7
set [ find default-name=ether8 ] name=ether8-RB750
/interface vlan
add interface=ether2 l2mtu=1584 name=vlan-2 vlan-id=1982
add interface=ether2 l2mtu=1584 name=vlan-1 vlan-id=2069
/interface bridge port
add bridge=bridge-LAN interface=ether3-virtual
add bridge=bridge-LAN interface=ether4-virtual2
add bridge=bridge-LAN interface=ether5-nas
add bridge=bridge-LAN interface=ether6-virtual4
add bridge=bridge-LAN interface=ether8-RB750
/ip address
add address=172.18.0.1/30 interface=vlan-1 network=172.18.0.0
add address=172.18.0.5/30 interface=vlan-2 network=172.18.0.4
/ip route
add distance=1 dst-address=10.3.17.0/24 gateway=172.18.0.2
add distance=1 dst-address=10.3.20.0/24 gateway=172.18.0.6

Site B (with CRS):

/interface ethernet
set [ find default-name=ether24 ] name=ether24-swiatlo
add interface=ether24-swiatlo l2mtu=1584 name=vlan-k vlan-id=2069
/ip address
add address=10.3.17.1/24 interface=bridge-LAN network=10.3.17.0
add address=172.18.0.2/30 interface=vlan-kato network=172.18.0.0

Site C (with RB2011):

/interface ethernet
set [ find default-name=sfp1 ] speed=100Mbps
/interface vlan
add interface=sfp1 l2mtu=1594 name=vlan1 vlan-id=1982
/ip address
add address=10.3.20.1/24 interface=bridge-LAN network=10.3.20.0
add address=172.18.0.6/30 interface=vlan1 network=172.18.0.4

What can I do more? On other environment RB2011 pushes around 200mbit+ with routing, queues, VLANs and EoIPs.

Thanks in advance!

2

Well, one thing I’m inferring from what config you did post is that you have the CRS set up with ports bridged (within Winbox, using the CPU) rather than using a master-port (switch chip based) setup. The CPU on the CRS is fairly limited (similar in capability to the RB2011), and the switch chip only has one single full-duplex Gigabit connection to the CPU. So if you have traffic from port-to-port on the CRS and are using bridged configuration rather than switch configuration, that traffic is hairpinning (running both ways) on that Gigabit lane to the CPU. Which would mean 100Mbps of transfer from ether1 to ether2 is actually consuming 100Mbps/100Mbps of traffic on the switch chip to CPU lane, and restricting the available traffic for the rest of the ports. Combining the switch chip lane with the CPU limitation, and I would expect to see no more than 200-300Mbps of total traffic (including port-to-port) running through a bridged CRS – you need to convert to the master-port switch chip configuration if you are in fact using the bridged setup.

Similarly, ether1-4 on the CCR1009 are on a switch chip, though there isn’t a CPU limitation on that unit – you still do have a limit of a total of 1Gbps traffic through all 4 Ethernet ports (ether1-4) if it has to hit the CPU (bridged, separate interfaces, etc).

Otherwise, your posted configs are too sparse for me to provide other suggestions.

3

Hi,
Thank you for your reply.

I have connection speed between A and B 100/100 mbit per sec and 50/50 between A and C, so even 100mbit will be satisfied. Unfortunately, speeds around 15mbit of bandwidth are not :slight_smile:

I would rather not posting whole configuration, because of public IP addresses, but one thing I’ve forgot to say - those vlans are not on any bridge - ip addresses are assigned directly to VLAN interfaces - routers are routing, not bridging.

Maybe upgrade’ing firmware will help?
I’m hopeless…

4

Well, ultimately, the CRS is a switch, and not designed specifically with routing functionality in mind.

Are you master-port (switching) ether1-23, or whichever ports are your LAN together? The interface name you referenced in your config was bridge-LAN, so I presumed that ether1-23 were members of a bridge (in CPU) rather than master-ported (switched) together, which would mean all Local Office B traffic passing through the CRS (in addition to all traffic passing to site A) was transiting the CPU on the CRS.

5

I think you should start with basic. Verify speed and duplex on all ports monitor cpu usage and look for interface errors.

Next step try to isolate the the issue with iperf.
Send a udp stream of 100Mb and look at the winboxes during that time so you will be able to see that 100Mb is entering the router and less is coming out. In a minute you will see which device is causing this.

Good luck.

6

Tried iperf tests - full bandwitdh is achievable even in one tcp session ;/ Now I dont know what is going on. HTTP transfer is a problem, iperf transmission not…
I think problem lies in CCR router, tried changing queue interface type, but no difference.
I have there also RB750 - will try with untagging vlans with him and transmit data with EoIP interfaces…

I regret not buing RB1100AH instead of CCR..