Slow performance of OSPF/MPLS network

Hello there,
hows going guys?
So, I’m having very terrible troubles with my small wireless setup. I can’t get proper speed on the end of the network (on R8). See picture bellow:
network-mot.png
This is a simple OSPF network which using VPLS tunnels, over MPLS, to deliver PPPoE service to the customer. Routing works alright, MPLS works alright, but bandwidth test has a strange behavior.
I’ve done some testing between MikroTiks, all on the TCP stream:

R1 send to R8 ~ 5Mbps
R1 send to R6 ~ 9Mbps
R1 send to R4 ~ 30Mbps

R4 send to R8 ~ 7Mbps
R4 send to R6 ~ 55Mbps

R6 send to R8 ~ 30Mbps

As you can see all links in the backbone have a 30M (one-way) connection at least. When I connect client through R8, dial up pppoe connection and try speedtest.net I get just 6M/6M (like on a bandwidth test result R1 to R8). I know that all my wifi links are not so “flash” (see attachment bellow), but I reckon I should get way more than bloody 6megs even with this setting.

Links:
link1.png
Additional info:
All links are running on 802.11, nv2 had a same behavior.
I wouldn’t blame the MPLS, but just to be sure:

  • all interfaces have L2MTU 1598 at least
  • MPLS interface MTU is in default 1508
  • I tried get the rid of the MPLS and use old-good eoip, but result on the client was same
    For communication between OmniTIKs and “HGs” (R4 ↔ R5 and R6 ↔ R7) I set up a virtualAP interface and running ospf via this one
    R3 & R5 are connected and powered from the OmniTIKs (R4 & R6) via eth02.
  • I must to set in OmniTIK /interface ethernet poe settings ether1-poe-in-long-cable=yes, otherwise did not power up the plugged device (cable to eth01 ~ 15m)
    I had a problem between R5 & R6, auto negotion indicated 10M connection so I disabled it and flicked manually to 100M. Apparently it works fine as you can see on the bandwidth test R4 to R6.
    I know that I should change ospf interface to non-broadcast one, because of Wi-Fi and multicast, but I don’t think so that the problem of slow connection is there.


    Configuration of MikroTiks:
R1 - RB 2011UAS
===============
/ip address
add address=10.1.255.1/32 interface=lobridge 
add address=10.1.1.1/30 interface=eth02

/routing ospf instance
set [ find default=yes ] router-id=10.1.255.1
/routing ospf interface
add authentication=md5 authentication-key=xyz interface=eth02 network-type=broadcast
add authentication=md5 authentication-key=xyz interface=lobridge network-type=broadcast passive=yes
/routing ospf network
add area=backbone network=10.1.0.0/16

/mpls ldp
set enabled=yes lsr-id=10.1.255.1 transport-address=10.1.255.1
/mpls ldp interface
add interface=eth02
/interface vpls
add name=tunel1 remote-peer=10.1.255.8 vpls-id=18:1

/ppp profile
add local-address=192.168.169.1 name=pppoe use-encryption=yes
/ppp secret
add name=client password=pwd123 profile=pppoe remote-address=192.168.169.100 service=pppoe
/interface pppoe-server server
add interface=tunel1 default-profile=pppoe one-session-per-host=yes


R2 - RB SXT G-5HPnD r2
======================
/ip address
add address=10.1.255.2/32 interface=lobridge
add address=10.1.1.2/30 interface=eth01
add address=10.1.2.1/30 interface=wlan1

/routing ospf instance
set [ find default=yes ] router-id=10.1.255.2
/routing ospf interface
add authentication=md5 authentication-key=xyz interface=eth01 network-type=broadcast
add authentication=md5 authentication-key=xyz interface=wlan1 network-type=broadcast
add authentication=md5 authentication-key=xyz interface=lobridge network-type=broadcast passive=yes
/routing ospf network
add area=backbone network=10.1.0.0/16

/mpls ldp
set enabled=yes lsr-id=10.1.255.2 transport-address=10.1.255.2
/mpls ldp interface
add interface=eth01
add interface=wlan1


R3 - RB SXT G-5HPnD r2
======================
/ip address
add address=10.1.255.3/32 interface=lobridge
add address=10.1.2.2/30 interface=wlan1
add address=10.1.3.1/30 interface=eth01

/routing ospf instance
set [ find default=yes ] router-id=10.1.255.3
/routing ospf interface
add authentication=md5 authentication-key=xyz interface=eth01 network-type=broadcast
add authentication=md5 authentication-key=xyz interface=wlan1 network-type=broadcast
add authentication=md5 authentication-key=xyz interface=lobridge network-type=broadcast passive=yes
/routing ospf network
add area=backbone network=10.1.0.0/16

/mpls ldp
set enabled=yes lsr-id=10.1.255.3 transport-address=10.1.255.3
/mpls ldp interface
add interface=eth01
add interface=wlan1


RB4 - OmniTIK UPA-5HnD
======================
/interface wireless
add name="wlan2" interface-type=virtual-AP

/ip address
add address=10.1.255.4/32 interface=lobridge
add address=10.1.3.2/30 interface=eth02
add address=10.1.4.1/30 interface=wlan2

/routing ospf instance
set [ find default=yes ] router-id=10.1.255.4
/routing ospf interface
add authentication=md5 authentication-key=xyz interface=eth02 network-type=broadcast
add authentication=md5 authentication-key=xyz interface=wlan2 network-type=broadcast
add authentication=md5 authentication-key=xyz interface=lobridge network-type=broadcast passive=yes
/routing ospf network
add area=backbone network=10.1.0.0/16

/mpls ldp
set enabled=yes lsr-id=10.1.255.4 transport-address=10.1.255.4
/mpls ldp interface
add interface=eth02
add interface=wlan2


R5 - RB SXT G-5HPnD r2
======================
/ip address
add address=10.1.255.5/32 interface=lobridge
add address=10.1.4.2/30 interface=wlan1
add address=10.1.5.1/30 interface=eth01

/routing ospf instance
set [ find default=yes ] router-id=10.1.255.5
/routing ospf interface
add authentication=md5 authentication-key=xyz interface=eth01 network-type=broadcast
add authentication=md5 authentication-key=xyz interface=wlan1 network-type=broadcast
add authentication=md5 authentication-key=xyz interface=lobridge network-type=broadcast passive=yes
/routing ospf network
add area=backbone network=10.1.0.0/16

/mpls ldp
set enabled=yes lsr-id=10.1.255.5 transport-address=10.1.255.5
/mpls ldp interface
add interface=eth01
add interface=wlan1
 

RB6 - OmniTIK UPA-5HnD
======================
/interface wireless
add name="wlan2" interface-type=virtual-AP

/ip address
add address=10.1.255.6/32 interface=lobridge
add address=10.1.5.2/30 interface=eth02
add address=10.1.6.1/30 interface=wlan2

/routing ospf instance
set [ find default=yes ] router-id=10.1.255.6
/routing ospf interface
add authentication=md5 authentication-key=xyz interface=eth02 network-type=broadcast
add authentication=md5 authentication-key=xyz interface=wlan2 network-type=broadcast
add authentication=md5 authentication-key=xyz interface=lobridge network-type=broadcast passive=yes
/routing ospf network
add area=backbone network=10.1.0.0/16

/mpls ldp
set enabled=yes lsr-id=10.1.255.6 transport-address=10.1.255.6
/mpls ldp interface
add interface=eth02
add interface=wlan2


R7 - RB SXT G-5HPnD r2
======================
/ip address
add address=10.1.255.7/32 interface=lobridge
add address=10.1.6.2/30 interface=wlan1
add address=10.1.7.1/30 interface=eth01

/routing ospf instance
set [ find default=yes ] router-id=10.1.255.7
/routing ospf interface
add authentication=md5 authentication-key=xyz interface=eth01 network-type=broadcast
add authentication=md5 authentication-key=xyz interface=wlan1 network-type=broadcast
add authentication=md5 authentication-key=xyz interface=lobridge network-type=broadcast passive=yes
/routing ospf network
add area=backbone network=10.1.0.0/16

/mpls ldp
set enabled=yes lsr-id=10.1.255.7 transport-address=10.1.255.7
/mpls ldp interface
add interface=eth01
add interface=wlan1


R8 - RB SXT G-5HPnD r2
======================
/ip address
add address=10.1.255.8/32 interface=lobridge
add address=10.1.7.2/30 interface=eth01

/routing ospf instance
set [ find default=yes ] router-id=10.1.255.8
/routing ospf interface
add authentication=md5 authentication-key=xyz interface=eth01 network-type=broadcast
add authentication=md5 authentication-key=xyz interface=lobridge network-type=broadcast passive=yes
/routing ospf network
add area=backbone network=10.1.0.0/16

/mpls ldp
set enabled=yes lsr-id=10.1.255.8 transport-address=10.1.255.8
/mpls ldp interface
add interface=eth01
/interface vpls
add name=tunel1 remote-peer=10.1.255.1 vpls-id=18:1

/ip bridge port
add interface=wlan1 bridge=br-client
add interface=tunel1 bridge=br-client

AIA (Any Ideas Anyone)?
Thank you for your time and effort.


Regards,
Petr

You always need to make 2 sets of tests - one TCP other UDP (and one more for bonus) multi TCP connection test. This way you will be able to narrow problem down.

hi,

at all routers turn down datarates manually until you reach 90-100% CCQ.

Test the max. MTU size you can send throught your VPLS tunnel.
Then modify MTU accordingly.

All wireless signals are bad to worse.

Agreed, the main problem here is that the wireless signal is bad.

You should ideally be -55 to -65

According to my test the best signal strenght is around -60dBm, but these are far away from this level, and your CCQ is not something good,in normal case it should be around 90-100% in both way

Your going to be a lot better of with NV2 then 802.11. It will help fix some rate flapping issues that your bound to see with 802.11 (rate flapping meaning the modulation changing up and down which will cause high veneration in latency and decrease your CCQ). Also your signal levels are not great but your SNR is mostly good. I would be most concerned about Link 1 which looks like chain 1 is having interference of some type. It also looks like chain1 of link 4 has a problem seeing that you are not showing Tx & Rx stats, I would guess that you have Tx or Rx turned off on one side or the other with that link.

Attached is a pic of what I generally consider a good connection (20MHz channel width that can pass just under 100Mbps TCP or UDP).
9km_MikroTik_BH.png

Hello guys,
sorry for late answer.

Thanks a lot for your replies! I consulted the problem with MikroTik support as well. Seems to be that the biggest issues is in the signal.
On link1, there is a potential interference in Fresler zone, will fix it with one “repeater solution”, 1x OmniTIK with 2x SXT HGs.
On link3, I’m going to put one more SXT HG and will do ptp with nv2.

I reckon it’s gonna take me couple of weeks so if is someone interested, be patient :slight_smile:

@joshaven
THX for your post and your time with it, yes I hadn’t “checked” both HT chains on link4…

@macgaiver
are you interested about UDP test results:
R1 send to R8 8Mbps/udp 6Mbps/tcp
R1 send to R6 17Mbps/udp 10Mbps/tcp
R1 send to R4 50Mbps/udp 50Mbps/tcp

R4 send to R8 13Mbps/udp 7Mbps/tcp
R4 send to R6 70Mbps/udp 56Mbps/tcp

R6 send to R8 45Mbps/udp 37Mbps/tcp

TA,
Regards,
Petr