Community discussions

 
User avatar
ringlet
newbie
Topic Author
Posts: 35
Joined: Sun Mar 07, 2010 9:24 pm
Location: Gdynia, Poland
Contact:

nv2 multilink problem

Mon Oct 15, 2012 2:54 am

Hi,
i am still looking for answer for this situation:

A <------> B <-------> C

A - rb750
B - rb711HnD (AP)
C - sextant (CPE)
between A and B is ubiquiti transparent bridge

btest A to B 1 tcp session 42 Mbit , 4 sessions 50Mbit
btest B to C 1 session 30-35 Mbit , 4 sessions 40-45 Mbit

and the riddle A to C only 19-23 Mbit

now im out of ideas how to resolve this

os 5.21 , different channels ,

what is wrong? why mt doest like 2 or more bridge in a line in a row?

please advise something .....

same proble in here:
http://forum.mikrotik.com/viewtopic.php?f=2&t=62377
 
dada
Member Candidate
Member Candidate
Posts: 245
Joined: Tue Feb 21, 2006 1:44 pm

Re: nv2 multilink problem

Mon Oct 15, 2012 2:04 pm

Hi,
i am still looking for answer for this situation:

A <------> B <-------> C

A - rb750
B - rb711HnD (AP)
C - sextant (CPE)
between A and B is ubiquiti transparent bridge

btest A to B 1 tcp session 42 Mbit , 4 sessions 50Mbit
btest B to C 1 session 30-35 Mbit , 4 sessions 40-45 Mbit

and the riddle A to C only 19-23 Mbit

now im out of ideas how to resolve this

os 5.21 , different channels ,

what is wrong? why mt doest like 2 or more bridge in a line in a row?

please advise something .....

same proble in here:
http://forum.mikrotik.com/viewtopic.php?f=2&t=62377
The A-C speed is measured using 1 or more concurrent TCP sessions?
What if CPU usage on B durin A-C test?
What is the RTT delay from A to C?
 
okoun
Frequent Visitor
Frequent Visitor
Posts: 98
Joined: Fri Feb 18, 2011 3:11 pm

Re: nv2 multilink problem

Mon Oct 15, 2012 5:10 pm

Hi!
I have less backbone on these points:

First NV2 2ms 802.11n MIMO 2x2, 300MB / s steady rate
Second NV2 2ms 802.11n MIMO 1x1, 150Mb / s rate stable
Third NV2 2ms 802.11n MIMO 1x1, 150Mb / s rate stable
4th 802.11a Nstreme 4000 framer, 54Mbps / s steady rate
5th NV2 2ms 802.11n MIMO 2x2, 130MB / s steady rate

The first to 5 UDP 40MB / s one TCP connection inexplicably flies from 3Mb / s to 10Mb / s very unstable! Unlike UPD test, 20 TCP 14-16Mb / s there was a jump in less than one.

Please RoS developers to look at optimizing the TCP NV2, it is very unfortunate that TCP has very little power compared to UDP. The Nstreme not noticed this problem, there can be measured on UDP 41Mb/sa pretty much the same about 39Mb / s on TCP.

Thank you very much, otherwise I am very satisfied with RoS.
 
uldis
MikroTik Support
MikroTik Support
Posts: 3425
Joined: Mon May 31, 2004 2:55 pm

Re: nv2 multilink problem

Mon Oct 15, 2012 5:26 pm

TCP on Nv2 works very well.
Make sure that you are not using bandwidth test between the wireless boards as the regular board can't generate so much TCP traffic - you need to push through the boards the traffic and generate that traffic on more powerful devices.
 
okoun
Frequent Visitor
Frequent Visitor
Posts: 98
Joined: Fri Feb 18, 2011 3:11 pm

Re: nv2 multilink problem

Mon Oct 15, 2012 5:42 pm

Hello, of course I have not done a test from the board. Testing was performed from the external device. specifically x86 PC where you can generate one direction without problem 80Mb/s TCP, I tested this between RB and x86 which is connected to a cable, there's a problem there. The problem has now come to the connection between the wireless NV2 Nstreme with this problem on wireless was not. I therefore conclude that the problem is on NV2.
I am writing because of this problem watching other colleagues, so I do not think this could cause any interference or faulty wireless card on my network.

thank you
 
sonny
Member Candidate
Member Candidate
Posts: 208
Joined: Fri Jan 28, 2005 5:14 pm
Location: Germany
Contact:

Re: nv2 multilink problem

Tue Oct 16, 2012 11:11 am

Hi,
this is the point were AP synchronisation comes into the play. :D (@Mikrotik when?)

Second at one Location use only AP mode and the next one only station mode.

Try different frequencies on the backhaul (some pairs doesn´t fit well) and use RF shields on back to back or nearby antennas.

then you will have bettter throughput and less rf interference.
Karl Sonnleitner
Senior Wireless-Expert
Restlesspowerbox - managed powersupply for Routerboards
 
rado3105
Member
Member
Posts: 480
Joined: Sat Jan 12, 2008 11:45 pm

Re: nv2 multilink problem

Tue Oct 16, 2012 9:42 pm

http://forum.mikrotik.com/viewtopic.php?f=7&t=66290


I wrote already about same problem..
 
okoun
Frequent Visitor
Frequent Visitor
Posts: 98
Joined: Fri Feb 18, 2011 3:11 pm

Re: nv2 multilink problem

Mon Oct 22, 2012 4:38 pm

It seems to me that here I do not believe anyone on the forum, so I put a test file to the PC machine into X86, calmly Which generates 100Mbps on TCP.
1. Test is on udp from PC to 5 jump which gives approximately around 80 to 90MB / s which is totally OK.
2. The test is disastrous. This is one of the TCP connection and not with bohybuje to 10Mb / s! What is the problem, please?
test2.jpg
You do not have the required permissions to view the files attached to this post.
 
uldis
MikroTik Support
MikroTik Support
Posts: 3425
Joined: Mon May 31, 2004 2:55 pm

Re: nv2 multilink problem

Mon Oct 22, 2012 5:14 pm

how fast is your x86 box? check the CPU usage on it.
How fast is the board that you send to the bandwidth test?
Try to increase the number of TCP connections to 20 or more
 
User avatar
ringlet
newbie
Topic Author
Posts: 35
Joined: Sun Mar 07, 2010 9:24 pm
Location: Gdynia, Poland
Contact:

Re: nv2 multilink problem

Mon Oct 22, 2012 7:29 pm

but thats the problem ! that nv2 cuts tcp to approximately 9-12 Mbit
same problem here ... after changing to nstreme tcp increases to 30-40 Mbit on machine

no 20 sessions - 1 session more bandwidth we need ! speedtest is only 1 session .....
 
okoun
Frequent Visitor
Frequent Visitor
Posts: 98
Joined: Fri Feb 18, 2011 3:11 pm

Re: nv2 multilink problem

Mon Oct 22, 2012 8:28 pm

Unfortunately, I still do not believe here in the forum.
I did a test that proves that in one generation TCP connection is absolutely no problem for this PC.
This is a test from PC to RB, which is connected to a cable 100Mb. So no NV2 and you can see that the connection is no problem.
test5.jpg
You do not have the required permissions to view the files attached to this post.
 
uldis
MikroTik Support
MikroTik Support
Posts: 3425
Joined: Mon May 31, 2004 2:55 pm

Re: nv2 multilink problem

Tue Oct 23, 2012 5:34 pm

Just made a lab test with 60db attenuation between the wireless boards (signal approx -50) with 2x2 Mimo and 20mhz channel width - data rates 130mbps. Wireless modes ap-bridge on the AP and station-bridge on the Client. On AP and on the Client ethernet interface is bridged with wireless interface.

RB1000---ethernet---RB711UA-5HnD/\/\wireless2x2(20mhz)/\/\RB711-5HnD---ethernet---RB800

Wireless board running v5.21 with RouterBoard firmware v3.0 and RB800/RB1000 RouterOS v6.0rc2.

Bandwidth test results:
UDP test between the RB1000<->RB800 is approx 97Mbps
TCP test 1 connection 88-92Mbps
TCP test 20 connections 90-93Mbps
 
dada
Member Candidate
Member Candidate
Posts: 245
Joined: Tue Feb 21, 2006 1:44 pm

Re: nv2 multilink problem

Tue Oct 23, 2012 6:22 pm

Just made a lab test with 60db attenuation between the wireless boards (signal approx -50) with 2x2 Mimo and 20mhz channel width - data rates 130mbps. Wireless modes ap-bridge on the AP and station-bridge on the Client. On AP and on the Client ethernet interface is bridged with wireless interface.

RB1000---ethernet---RB711UA-5HnD/\/\wireless2x2(20mhz)/\/\RB711-5HnD---ethernet---RB800

Wireless board running v5.21 with RouterBoard firmware v3.0 and RB800/RB1000 RouterOS v6.0rc2.

Bandwidth test results:
UDP test between the RB1000<->RB800 is approx 97Mbps
TCP test 1 connection 88-92Mbps
TCP test 20 connections 90-93Mbps
If the problem we are facing is realy caused by TCP window size negotiation mechanism then the crutial question is: What RTT (ping time) between the ends do you have in your lab case? I would expect rather low value since you have only one client connected and you are using 2x2 MIMO (so you have 130mbps modulation speed) and you have no other links included. This is far from the real case scenario, IMHO. Btw - does the 6.x use the same TCP Window default size as 5.X?

If you have an AP with NV2 with non-trivial amount of clients you will easily get 20+ms RTT just between the client and the AP. In our case it could mean that the RTT to our network border increases several times. And now imagine you will use NV2 on PtP link (because the MIMO allows you to create higher capacity link and nstreme doesn't like 802.11N and multichain) to connect the AP...

If the customer on such AP is allowed to use higher max speeds it could lead to the problem that it will not be able to reach the MAX speed using just one TCP connection (for example to generally any Internet speedmeter (perhaps except the http://www.speedtest.net)).

If you will google enough you can find articles related to problem of TCP window size and tables which shows you what would be TCP speed depending on the end to end transport delay. The speed goes down with higher delays, It can be improved with changing the OS system TCP window default size...

http://bradhedlund.com/2008/12/19/how-t ... nce-links/

So I think Mikrotik should invest some effort to decrease the NV2 latency. And I personally would be very happy if Mikrotik work on nstreme support for MIMO. It would easily allowed to build faster low latency uplinks...
 
rado3105
Member
Member
Posts: 480
Joined: Sat Jan 12, 2008 11:45 pm

Re: nv2 multilink problem

Tue Oct 23, 2012 8:22 pm

in lab was used 2x v6 firmware, maybe this problem is not there...
Yes nstreme on N cards would be great, also dual nstreme.
 
User avatar
honzam
Forum Guru
Forum Guru
Posts: 2287
Joined: Wed Feb 27, 2008 10:27 pm
Location: Czech Republic

Re: nv2 multilink problem

Wed Oct 24, 2012 9:15 am

RB1000---ethernet---RB711UA-5HnD/\/\wireless2x2(20mhz)/\/\RB711-5HnD---ethernet---RB800
Hello Uldis.
Can you make the same test in this configuration?

RB1000---ethernet---RB711UA-5HnD/\/\wireless2x2(20mhz)/\/\RB711-5HnD---ethernet(rb450g)---RB711UA-5HnD/\/\wireless2x2(20mhz)/\/\RB711-5HnD ---RB800
LAN, FTTx, Wireless. ISP operator
 
User avatar
ringlet
newbie
Topic Author
Posts: 35
Joined: Sun Mar 07, 2010 9:24 pm
Location: Gdynia, Poland
Contact:

Re: nv2 multilink problem

Wed Oct 24, 2012 10:39 am

I'm really happy that not only I have this problem, and that in the end the team Mikrotik deal with this problem. Maybe something will finally be explained.
 
Holekm
just joined
Posts: 9
Joined: Wed Oct 24, 2012 8:34 pm

Re: nv2 multilink problem

Mon Oct 29, 2012 12:05 pm

I have the same problem - 433AH with 3xAtheros 5GHz 40MHz and about 3 clients per card with RB411. With torrent i can get about 35Mbps or more, with tcp - max 6Mbps per sesson. When using downloader or DownThemAll (Firefox Addone) - there is more with multiple connections, but this not resolve the problem of BAD ONE TCP CONNECTION PERFORMANCE. With Nstreme performance is much better but there are disconnections and stability problems so i can't use it. I was testing many configurations with many RouterOS versions and the problem still persist. Problem have nothing to do with noisy enviroment or wheather or bad antenas (ccq 99/99) or settings. Problem is the same when one or many cilents are connected with one card on AP. Only TDMA period size have slighty influence on performance but its 6Mbps with 1ms TDMA and 10Mbps with 2ms, but then - if i make 3ms or more performace stays on 10Mbps.
 
User avatar
ringlet
newbie
Topic Author
Posts: 35
Joined: Sun Mar 07, 2010 9:24 pm
Location: Gdynia, Poland
Contact:

Re: nv2 multilink problem

Tue Oct 30, 2012 8:42 am

so Mikrotik team, will You try to help us? Will You investigate this issue ? I see more and more posts like that in local forum in Poland ( trzepak.pl ) also on ubnt forum and here on Mikrotik forum , so I gues it is not only my problem...
 
User avatar
honzam
Forum Guru
Forum Guru
Posts: 2287
Joined: Wed Feb 27, 2008 10:27 pm
Location: Czech Republic

Re: nv2 multilink problem

Tue Oct 30, 2012 12:27 pm

so Mikrotik team, will You try to help us? Will You investigate this issue ? I see more and more posts like that in local forum in Poland ( trzepak.pl ) also on ubnt forum and here on Mikrotik forum , so I gues it is not only my problem...
+1
The same problem on Czech forum (ispforum.cz)

Mikrotik help us
LAN, FTTx, Wireless. ISP operator
 
Ciambot
Member Candidate
Member Candidate
Posts: 144
Joined: Thu Jan 22, 2009 5:22 pm

Re: nv2 multilink problem

Sat Nov 03, 2012 9:19 pm

I've same problem. +1
 
User avatar
ringlet
newbie
Topic Author
Posts: 35
Joined: Sun Mar 07, 2010 9:24 pm
Location: Gdynia, Poland
Contact:

Re: nv2 multilink problem

Sat Nov 03, 2012 9:30 pm

how to resolve this problem:

if U have p2p bidge in 2x2 and then MT with AP - U should change 2x2 on 1x1 and change nv2 to nstreme on the bridge
then U can leave nv2 on AP

nv2 should be used only at the ends in AP , no nv2 with p2p bridge ...

we tried everything nv2 is dooing "something" with TCP frames :(
 
denis
Frequent Visitor
Frequent Visitor
Posts: 71
Joined: Tue Sep 20, 2005 9:27 pm
Location: Serbia

Re: nv2 multilink problem

Mon Nov 05, 2012 12:10 pm

Hello Uldis.
Can you make the same test in this configuration?

RB1000---ethernet---RB711UA-5HnD/\/\wireless2x2(20mhz)/\/\RB711-5HnD---ethernet(rb450g)---RB711UA-5HnD/\/\wireless2x2(20mhz)/\/\RB711-5HnD ---RB800
I did something like this. Instead of RB1000 and RB800 I had dual XEON servers ... much like this:

Image

Laptop in the above situation was just for the link performance test.
The above setup considers using VPLS based transparent bridge. With the routed setup it just gets worse. With WDS I got better results than with routed network, but less than with VPLS tunnel, so I ended up using VPLS. If I add a third wireless hop, at the endpoint I get 45 - 50 Mbps of TCP. Each wireless hop is running on top of NV2. NStreme did show a better latency, but as the same had less troughoutput by at least 15%. Firewall and conntrack turned off on all RB435G's. Latest tested ROS version was 5.14.

I hoped to see at least 100 Mbps of TCP end-to-end, but as you can see I ended up with max 76 Mbps, which is rather disappointing and discouraging. It is a transparent bridge and it should be acting as a simple cable ... but it doesn't.

Bottom line is, I could not get Mikrotik to forward TCP traffic as it should over multi-hop 802.11n wireless. As the same, I cannot find anyone in this forum that bragged about succeeding to do so.

So, what is the problem with (TCP) forwarding?
 
Ciambot
Member Candidate
Member Candidate
Posts: 144
Joined: Thu Jan 22, 2009 5:22 pm

Re: nv2 multilink problem

Tue Nov 06, 2012 11:37 am

how to resolve this problem:

if U have p2p bidge in 2x2 and then MT with AP - U should change 2x2 on 1x1 and change nv2 to nstreme on the bridge
So, back to 3 years ago. :?
 
User avatar
ringlet
newbie
Topic Author
Posts: 35
Joined: Sun Mar 07, 2010 9:24 pm
Location: Gdynia, Poland
Contact:

Re: nv2 multilink problem

Tue Nov 06, 2012 11:54 am

how to resolve this problem:

if U have p2p bidge in 2x2 and then MT with AP - U should change 2x2 on 1x1 and change nv2 to nstreme on the bridge
So, back to 3 years ago. :?
or... put some fiber in to ground like we did !
there is more , You can always buy professional radioline ...

no good news ...
 
rado3105
Member
Member
Posts: 480
Joined: Sat Jan 12, 2008 11:45 pm

Re: nv2 multilink problem

Tue Nov 06, 2012 2:39 pm

Interesting that no word from mikrotik....
 
zyzelis
Member Candidate
Member Candidate
Posts: 212
Joined: Sun Apr 08, 2012 9:25 pm

Re: nv2 multilink problem

Wed Nov 07, 2012 11:15 pm

As always.....
So the decision is to wait better support from Mikrotik before buying wireless.
And please Normis don't waste your time and do not answer
 
JorgeAmaral
Trainer
Trainer
Posts: 199
Joined: Wed Mar 04, 2009 11:53 pm
Location: /ip route add type=blackhole

Re: nv2 multilink problem

Sat Nov 10, 2012 2:58 pm

I don't know where your problems are, but i will give my experience with NV2.

In the beginning (< 5.14 ) it was a headache to have a stable high throughput link.

Since the appearance of station-bridge mode I stooped using vpls.

But with +5.20 versions, running on 680Mhz cpu routerboards I say it´s amazing and Mikrotik finally got it really good.

Just one of my links running 5.20:

<laptop with iperf > < NV2 mode: Bridge A > 4KM link < mode: station-bridge A > ethernet cable < mode: station-bridge B> 1km link < NV2 mode:Bridge B > <laptop with iperf>

Running NV2 with 20Mhz channel, NV2 period-size 2ms, cell-radius 10km, one link running on 5640Mhz, other running on 5700Mhz.

I have CCQ of 100% on both links and both directions and signals are around -65dBm, reduced Tx-power to achieve these values.

Iperf tests with TCP and only one stream from laptop to laptop I get 96Mbps throughput with latency jumping between 7ms and 18ms.

The only reliable way to discover real TCP throughput of the links is with external devices (laptops or routerboards). I prefer iperf and playing with window size, multiple connections, etc..., then a simple ftp server and download an ISO.

Have a good weekend :-)

Edit:

I forgot to mention that with RB411 versions with 680Mhz cpu, one single tcp stream achieves 96Mbps on a 10/100M ethernet interface.

With RB433GL one single stream achieves only 75Mbps of TCP, but with 5 tcp connections on iperf it reaches 216Mbps average on 5 minute test on a 1,5km link. These devices have some limitation on single stream, but has i dont provide more that 10Mbps for each customer, it´s not critical in our case.
 
denis
Frequent Visitor
Frequent Visitor
Posts: 71
Joined: Tue Sep 20, 2005 9:27 pm
Location: Serbia

Re: nv2 multilink problem

Sat Nov 10, 2012 10:42 pm

And there you go ... That's the reason why this forum exists in the first place and I must admit I was kind of hoping someone would show up and say "Dudes, it's working for me just fine ..." and suggest a setup of his own.

Tnx JorgeAmaral. I'll try your suggestion tonight on a testing link and see if it works for me as well.

Regards!
 
denis
Frequent Visitor
Frequent Visitor
Posts: 71
Joined: Tue Sep 20, 2005 9:27 pm
Location: Serbia

Re: nv2 multilink problem

Sun Nov 11, 2012 10:16 pm

JorgeAmaral, u just earned some karma from me!

Your suggestion proved to be working just fine and, for some reason, better than VPLS based bridge. This works for me, so tnx and regards!
 
uldis
MikroTik Support
MikroTik Support
Posts: 3425
Joined: Mon May 31, 2004 2:55 pm

Re: nv2 multilink problem

Mon Nov 12, 2012 10:11 am

JorgeAmaral, u just earned some karma from me!

Your suggestion proved to be working just fine and, for some reason, better than VPLS based bridge. This works for me, so tnx and regards!
what results you are getting now?
What configuration you are using now?
 
denis
Frequent Visitor
Frequent Visitor
Posts: 71
Joined: Tue Sep 20, 2005 9:27 pm
Location: Serbia

Re: nv2 multilink problem

Tue Nov 13, 2012 9:55 am

Well uldis, unfortunately it appears I got excited too soon, after all.

When I did a bw test (dual Xeon to dual Xeon with iperf), while there was no live traffic running over this link I managed to push 165 Mbps. Comparing to 75Mbps which I had, this was more than I expected.

However, yesterday evening, 15 minutes after I released the real traffic over this link, suddenly it collapsed and max I could get went down to 55 Mbps again. So ... I ended up where I was few days back.

I did a test on these transparent bridge setups:
1). MPLS/VPLS tunnel on top of NV2 wireless, inside the bridge (rstp) with ETH
2). WDS on top of NV2 wireless, inside the bridge (rstp) with ETH
3). bridge/station bridge NV2 inside the bridge (rstp) with ETH

Firewall and contrack turned off in each setup.
On VPLS setup, I got 75Mbps, WDS was like 15-20% slower. Bridge/station bridge, as I said gave 165 Mbps.
Situation is as presented in the drawing of my previous post:
Image
I also did a test with v6.0rc2 + custom channels (30 Mhz channel width). On 17 km wireless link (P2P) I got incredible results. Ping 4 - 7 ms, while pushing 190 Mbps TCP (dual Xeon to laptop). R52Hn + 29 dBi dual dish, CCQ 98/97%, signal level -63/-64, noise level 58dBi. You cannot get better than this. But ... It appears like transparent bridge is not transparent enough and as soon as it starts transporting a real traffic it collapses.

So, after 7 years of my work with mikrotik, either I do not have enough knowledge to set this or mikrotik is incapable of forwarding TCP over multihop 802.11n links.

Very, very, very frustrating. Sorry if I mislead anyone, and even more sorry for misleading my self.

Btw, karma for JorgeAmaral still stands ... he tried to help.
 
rado3105
Member
Member
Posts: 480
Joined: Sat Jan 12, 2008 11:45 pm

Re: nv2 multilink problem

Tue Nov 13, 2012 9:51 pm

Yes it seems that mikrotik has problem with tcp forwarding. Please give us back nstreme and dual nstreme on N cards. Or make better N cards....
 
uldis
MikroTik Support
MikroTik Support
Posts: 3425
Joined: Mon May 31, 2004 2:55 pm

Re: nv2 multilink problem

Wed Nov 14, 2012 12:35 pm

denis, maybe you are experiencing interference on that tower in the middle - first link disturbs the second link. As it might happen even if you are using distant wireless frequencies from each other. That is why we recommend you to move the antennas more further away from each other on the tower.
Also please test using v5.21 or 5.22rc1 when doing the tests.
 
denis
Frequent Visitor
Frequent Visitor
Posts: 71
Joined: Tue Sep 20, 2005 9:27 pm
Location: Serbia

Re: nv2 multilink problem

Wed Nov 14, 2012 2:53 pm

Uldis,

Oh, wouldn't it be great if that was the case. Unfortunately, it's not. I am trying to solve this issue for over a year now. And beleive me I tried all I could. In this particular case, the antennas you are suspecting of are 5 meters apart one from the other. The antennas are pointing in opposite directions (180 degrees) and there is a sort of a shield (a structure) between them. Further, the antennas are parabolic dishes - which gives additional isolation from noise. And yes, as you said, the freqs are far apart (5100 on one side / 5900 on the other ... Dear "superchannel").

Here is what I observed in the tests I had in past couple of days:
The best test results I got using either VPLS/bridge or station-bridge/bridge setups, on top of NV2. As long as I run a point-to-point bw test, all looks good. However, very moment I release a real traffic and the number of packet flowing trough the bridge starts increasing, the link capacity starts dropping. I end up with 1/3 of originally tested capacity.

If you Google it up, you will find a lot of docs confirming a problem in forwarding TCP packets over multi-hop networks in 802.11n, and therefore confirming my above observation. Now, the protocol running on top of 802.11n and/or transparent bridge (per some Mikrotik related articles) should solve this problem. But ... they don't. At least transparent bridge is not.

So, I did another test using Nstreme (plus station-bridge/bridge) instead of NV2, and you know what ... nstreme seems to be solving the problem. The link is forwarding traffic. Not as much as on the tests with NV2, but much more than with NV2 in real-life situation. I got about 130 Mbps which is almost double of NV2. Ping is better: 4 - 8 ms end-to-end. BUT (and there we go again) ... Nstreme is not stable on 802.11n MIMO. I get disconnects every 5 - 15 minutes, with a message "disconnected, too many poll timeouts" (as written a lot of times in this forum).

Bring up your own conclusion, but there you have it: NSTREME (almost) works and NV2 doesn't. So, uldis, I will have to join the group of people yelling at MT to solve the problem for I am convinced now that the protocol is the solution and I have my bets on Nstreme (providing you solve the problem with frequent disconnects).

Btw, in defense of Mikrotik ... the other vendors have the same or worse problems (UBNT confirmed to have the same issues). Not that it's much of a comfort, but ... 8)

I most sincerely hope that the above observation will help MT team to provide us with a solution to multihop links (I have a full faith in you guys!).

Oh, yeah, v5.21 ... I'll give it a shot and if any changes I'll let you know. So far the tests I did were done on v5.18 and v.6.0rc2.


-------
P.S.:
One other thing, a bit off topic here but ... There's something wrong with vpls/mpls in MT ROS 6.0rc2. I'll open a discussion on that one in the appropriate forum topic.
 
denis
Frequent Visitor
Frequent Visitor
Posts: 71
Joined: Tue Sep 20, 2005 9:27 pm
Location: Serbia

Re: nv2 multilink problem

Fri Nov 16, 2012 10:31 am

As I promised, I did a test with 5.21 and ... it's the same. However, and I hope it will help some of you, I did find a temporary solution.

I was playing around with nstreme to see if I can get it stable on 802.11n and made it. Here is a setup of my current p2p links:

- MT ROS version: 6.0 rc3
- mode: bridge (ap side) / station bridge (client side)
- protocol: nstreme
- framer policy: best fit
- framer limit: 4000
- vpls tunnel (LDP on loopback IP address)
- bridge ports: ETH+vpls

Two hops in a chain: 17 + 5 km. The link is stable for over 15 hours now. I get an average ping of 8 ms and 130 Mbps of real traffic. Here and there I get a single ping jumps to 32 ms, but nothing serious. Sounds good? Well, not quite so. I'm experiencing problems with multimedia content (youtube, skype ...). Maybe it related to HT AMPDU priorities as someone mentioned it in this forum. Honestly, I do not know. I left them as per default. Anyone had similar problems or have a suggestion?
 
Ciambot
Member Candidate
Member Candidate
Posts: 144
Joined: Thu Jan 22, 2009 5:22 pm

Re: nv2 multilink problem

Mon Nov 19, 2012 2:25 pm

Denis, how much is your ccq? (both links and both protocols)
 
User avatar
ringlet
newbie
Topic Author
Posts: 35
Joined: Sun Mar 07, 2010 9:24 pm
Location: Gdynia, Poland
Contact:

Re: nv2 multilink problem

Mon Nov 19, 2012 4:15 pm

nstreme is the only answer

try to change 2x2 on 1x1 HT it should prevent from disconnects every 5-15 minuts
Last edited by ringlet on Mon Nov 19, 2012 9:18 pm, edited 2 times in total.
 
Ciambot
Member Candidate
Member Candidate
Posts: 144
Joined: Thu Jan 22, 2009 5:22 pm

Re: nv2 multilink problem

Mon Nov 19, 2012 8:59 pm

I updated all links to Ros 5.21, and set modulation to obtain 100% ccq. Bandwidth has increased about 20Mb, and now is stable (not fluctuates a lot, like before).
I don't think the multihop problem is resolved, but I improved my situation.
I need to do other tests in "real life" environment.
 
denis
Frequent Visitor
Frequent Visitor
Posts: 71
Joined: Tue Sep 20, 2005 9:27 pm
Location: Serbia

Re: nv2 multilink problem

Tue Nov 20, 2012 10:32 am

Denis, how much is your ccq? (both links and both protocols)
On a 17 km link, CCQ is 100/92%. On a 5 km link it's 100/100%.
nstreme is the only answer

try to change 2x2 on 1x1 HT it should prevent from disconnects every 5-15 minuts
I second that. Nstreme is the way, but I managed to suppress disconnects by forcing framer_limit=4000 (give some karma if it works for you too). In past 4 days, I had only two disconnects, for a second or so.

I updated all links to Ros 5.21, and set modulation to obtain 100% ccq. Bandwidth has increased about 20Mb, and now is stable (not fluctuates a lot, like before).
I don't think the multihop problem is resolved, but I improved my situation.
I need to do other tests in "real life" environment.
As for the "real life" test ... prepare yourself for a shock once the number of packages starts raising.
And, yes, multihop still remains unresolved.

MT team ... any propositions, workarounds, workabouts? Anything?



P.S.: It's kind of comforting to see someone else is trying to solve (discuss on the solution to) this problem.
 
karlos
Frequent Visitor
Frequent Visitor
Posts: 81
Joined: Sun Jan 16, 2011 10:46 am

Re: nv2 multilink problem

Tue Nov 20, 2012 11:19 am

I am sorry to report nearly the same problem.

If we use more than one hop on nv2 there is unexplained loss of transfer speed. We have tested it on

link (Dual nv2/40MHz channel) -52/-52dB 300/300Mbit 98/99CCQ
and next hop
link (Single nv2/40MHz channel) -49/-48dB 150/150Mbit 85/100CCQ
and at last the AP
Mutiuser AP on nv2 using 20MHz channel link speeds 58.5-65Mbps when on load signals around -60dB

and final troughput was 3-5Mbit download speed :-(

so we decided to switch to (5GHz-A) 802.11a mode, now clients running on 54Mbit and speeds always over 22Mbit

And on some another links we found faster speeds using nv2+nstreme+802.11a comparing to nv2+nv2+something
I am not sure whats happening, but it looks like the nv2 dont like to be in row. Looks like "SURGE" problem.

Its curious, but the same configuration never works same on various locations. And its not depending on noise, lenght or anything that can be meassured.

We would like to see more consistant results on same configuration. We dont like to "tune" every link we made, because there is no way to test it remotly if it works. We always need to stand up, use notebook on-site and test it. Even link that looks great on parameters like signal, ccq or linkspeed can be realy slow in real application, and the situation changes in time.
 
karlos
Frequent Visitor
Frequent Visitor
Posts: 81
Joined: Sun Jan 16, 2011 10:46 am

Re: nv2 multilink problem

Tue Nov 20, 2012 11:30 am

Dear ULDIS:
we too are trying to make some tests in "LAB" but it looks like it is more and more useless.

MK links are working great on table but if we put it outsides, something changes...... and its not so great like on table before

In opposite to this, if we try to test some "PRO" links 10-80GHz it is always pretty difficult to make run in "LAB" situation. But if we put it outsides, it works perfect.

So, based on this experience, we will test all links outsides, coz testing it in "LAB" will not help us at all. And i will recommend it to you too. Please make some test link outsides, try some setups, and give us results. If you will we can make tests for your configuration too, ane compare it. This will give you and idea how your products work in real, not in MUM or LAB situation.

Thank you
 
User avatar
honzam
Forum Guru
Forum Guru
Posts: 2287
Joined: Wed Feb 27, 2008 10:27 pm
Location: Czech Republic

Re: nv2 multilink problem

Tue Nov 20, 2012 11:59 am

MT support does not answer again? No idea? NO advice?
LAN, FTTx, Wireless. ISP operator
 
karlos
Frequent Visitor
Frequent Visitor
Posts: 81
Joined: Sun Jan 16, 2011 10:46 am

Re: nv2 multilink problem

Mon Nov 26, 2012 6:00 pm

MT support does not answer again? No idea? NO advice?
The only advice is : "If you dont answer, the problem vanishes" :lol:

I am really sorry. If we will help resolve problem, and there is no interest or will.
 
denis
Frequent Visitor
Frequent Visitor
Posts: 71
Joined: Tue Sep 20, 2005 9:27 pm
Location: Serbia

Re: nv2 multilink problem

Wed Nov 28, 2012 9:39 am

It appears so, karlos. Doesn't it?

Aside that MT team is being silent, I see over 2000 reads of this topic, yet there are only a couple of us exchanging experiences and taking active participation in discussion. Is it because maybe only us are having this problem? Nobody else noticed anything? Or perhaps it a secret and that is why everyone else is keeping quiet?

Anyway you choose it, it's pretty frustrating.

So far, in this topic, we pointed out that NV2 in multihop MIMO 802.11n just doesn't work at all. We proved that for some reason nstreme appears to get the job done. Than again, nsteme and 802.11n don't play well, so you get frequent disconnects. Why is it that nstreme works and NV2 doesn't I honestly do not know. But, and this one goes to MT team, solve frequent disconnects on nstreme and you have solved multihop problem. Hopefully it will also solve the problem of passing multimedia traffic as well (see my earlier posts in this topic).

And back to the last post of karlos ... Pssssst! Maybe it didn't see us and will go by ... 8)
 
karlos
Frequent Visitor
Frequent Visitor
Posts: 81
Joined: Sun Jan 16, 2011 10:46 am

Re: nv2 multilink problem

Wed Nov 28, 2012 2:44 pm

2 denis: Case of disconnects......

i found that on longer distances (1km and more) or in noisy area works single chain 802.11n nv2 as the best. I have also reported that even if 40MHz MiMO link shows 300/300Mbit and Btest shows great numbers, in real the "slower" 40HMz SiSo (just chain 0) works faster.

So we had disconnect issues on one cca 8km link. We put just 40MHz channel, one chain and set hw restries to 15. And i am now looking at the link at it runs 105days and it is registered 14d 04hours.... so it is not fully ok, but one disconnect in 14days seems to be acceptable on lowcost link.

Modulation 108,121,5-135Mbit, CCQ cca 75-87%, signal -56/-57dB, real traffic going trough 27,31Mbit download and 3,97Mbit upload as 1hour average. I saw some 5min peaks over 35Mbit, and it was still good accesible, so not on the limit. Ping 4,96-29,72ms, average 7,21ms measured from monitoring point, not just a latency of this link.........so looks ok
 
Lakis
Forum Veteran
Forum Veteran
Posts: 701
Joined: Wed Sep 23, 2009 7:52 pm

Re: nv2 multilink problem

Sat Dec 01, 2012 12:13 pm

I am sorry to report nearly the same problem.

If we use more than one hop on nv2 there is unexplained loss of transfer speed. We have tested it on

link (Dual nv2/40MHz channel) -52/-52dB 300/300Mbit 98/99CCQ
and next hop
link (Single nv2/40MHz channel) -49/-48dB 150/150Mbit 85/100CCQ
and at last the AP
Mutiuser AP on nv2 using 20MHz channel link speeds 58.5-65Mbps when on load signals around -60dB

and final troughput was 3-5Mbit download speed :-(

so we decided to switch to (5GHz-A) 802.11a mode, now clients running on 54Mbit and speeds always over 22Mbit

And on some another links we found faster speeds using nv2+nstreme+802.11a comparing to nv2+nv2+something
I am not sure whats happening, but it looks like the nv2 dont like to be in row. Looks like "SURGE" problem.

Its curious, but the same configuration never works same on various locations. And its not depending on noise, lenght or anything that can be meassured.

We would like to see more consistant results on same configuration. We dont like to "tune" every link we made, because there is no way to test it remotly if it works. We always need to stand up, use notebook on-site and test it. Even link that looks great on parameters like signal, ccq or linkspeed can be realy slow in real application, and the situation changes in time.
I talk about this not so long time ago it all come to this "NV2 nature works that way" except MT dont come out with something new :)
definitely I m so happy with nstreme
but when it comes to p2mp (something like Omnitik+SXTs for CPE...) boy how much NV2 work so well for me


On your first link try lowering your data rate from 300/300 something like 180/180
try testing UDP also test from bout directions
when testing TCP add 10 or more connections
try lowering ur signal >-58
on second link (this one 150/150) try using nstreme (u can also try selecting 5ghz-A)
also try nstreme+nstreme I would love to see your results
best wishes..
 
rado3105
Member
Member
Posts: 480
Joined: Sat Jan 12, 2008 11:45 pm

Re: nv2 multilink problem

Sun Dec 02, 2012 12:51 pm

8ms average is prety much and latency is the biggest problem of NV2....
 
karlos
Frequent Visitor
Frequent Visitor
Posts: 81
Joined: Sun Jan 16, 2011 10:46 am

Re: nv2 multilink problem

Tue Dec 04, 2012 12:02 pm

8ms average is prety much and latency is the biggest problem of NV2....
Please read. I have written that "not just a latency of this link"

Latency on the link alone is much smaller. 8ms is from our office trough, MyPC - Router - 700m 80GHz link - switch - 1km 24GHz link - switch - OSPF router - BGP router - 140km optical cable - BGP router - 120km otical cable - BGP router - switch - 2km 10GHz link - router - mikrotik link - final destination


..... so dont talk about too big latency please
 
bbs2web
Member Candidate
Member Candidate
Posts: 198
Joined: Sun Apr 22, 2012 6:25 pm
Location: Johannesburg, South Africa
Contact:

Re: nv2 multilink problem

Tue Dec 04, 2012 10:57 pm

This is similar to a problem I posted (http://forum.mikrotik.com/viewtopic.php?t=67835) where I'm testing RB433GL with R52Hn cards in a lab environment prior to installation.

I get excellent throughput in either direction using 5GHz-N using the nv2 wireless protocol but full duplex TCP is terrible. Speed results are based on traffic generated through ether ports of RB433GLs.

Setup:
Linux ----- RB433GL ========= RB433GL ----- Linux

Speed results:
Chained MIMO (300/300 with 100% CCQ:
UDP half duplex: 159Mbps/151Mbps (this was using RB bwtest)
UDP full duplex: 116Mbps/92Mbps
TCP half duplex: 173Mbps/151Mbps
TCP full duplex: 40Mbps/15Mbps

Have tried full duplex with 5GHz-A, nstream, 802.11 using both independent AP-station links using OSPF to separate the traffic and Nstream Dual. I really believe there to be a limitation on RB when trying to get it to run two TCP streams at the same time, regardless of the protocol.

I'll be providing feedback on moving the 2nd R52Hn in to a 2nd RB433GL and doing OSPF between them:
ie: Linux 1Gbps - ether1 RB433GL R52Hn wlan1 ---------- wlan1 R52Hn RB433GL ether1 - 1Gbps Linux
                         ether2                                     ether2
                         ether2                                     ether2
                         RB433GL R52Hn wlan1 ---------- wlan1 R52Hn RB433GL
PS: I've disabled connection tracking and can confirm that the CPU isn't hitting limitations. CCQ also still shows 100% and frequencies are 260MHz apart. Get identical full duplex transfer rates when I disable the 2nd chain on the current links although full duplex UDP and uni directional TCP drops by about half.
 
Ciambot
Member Candidate
Member Candidate
Posts: 144
Joined: Thu Jan 22, 2009 5:22 pm

Re: nv2 multilink problem

Mon Dec 17, 2012 12:26 am

What I see in my network...

Nv2 multilink problem starts when at least one link carries about over 1/2 or 3/4 its maximun bandwidth.
In that moment the single TCP connection slows down, although bandwidth is available.

Ex:
I've 5 links nv2, one of this is full at 80% by real traffic.
if I check the bandwidth with btest 20TCP connections (or UDP), I can reach maximun bandwith of the 5 links.
But if I execute btest setting TCP 1 connection, I get poor performance.

Therefore, in presence of many nv2 links, we need to oversize them, otherwise customers will have slow internet connection.
 
Ciambot
Member Candidate
Member Candidate
Posts: 144
Joined: Thu Jan 22, 2009 5:22 pm

Re: nv2 multilink problem

Thu Dec 20, 2012 3:55 pm

Nv2 multilink problem starts when at least one link carries about over 1/2 or 3/4 its maximun bandwidth.
Correction: the problem starts when the occupied bandwidth reaches 1/3-1/4 of total.
More traffic on the link (or links) and more the problem grows.
 
denis
Frequent Visitor
Frequent Visitor
Posts: 71
Joined: Tue Sep 20, 2005 9:27 pm
Location: Serbia

Re: nv2 multilink problem

Sat Dec 22, 2012 3:58 pm

Correction: the problem starts when the occupied bandwidth reaches 1/3-1/4 of total.
More traffic on the link (or links) and more the problem grows.
It's rather connected to the number of packages flowing trough. Check my earlier posts.
As for the "real life" test ... prepare yourself for a shock once the number of packages starts raising.
 
Ciambot
Member Candidate
Member Candidate
Posts: 144
Joined: Thu Jan 22, 2009 5:22 pm

Re: nv2 multilink problem

Sat Dec 22, 2012 7:18 pm

Correction: the problem starts when the occupied bandwidth reaches 1/3-1/4 of total.
More traffic on the link (or links) and more the problem grows.
It's rather connected to the number of packages flowing trough. Check my earlier posts.
As for the "real life" test ... prepare yourself for a shock once the number of packages starts raising.

Right, this time I achieved these results with latest version of RouterOS and all links with 100% CCQ.
:(
 
User avatar
DogHead
Member Candidate
Member Candidate
Posts: 194
Joined: Thu Jan 03, 2008 9:36 pm
Location: Anywhere you want me to be

Re: nv2 multilink problem

Thu Jan 10, 2013 6:27 pm

We are seeing similar types of issues with the RB433AH and R52Hn in point to point to multipoint configurations. I think that there may be even more going on here than just protocol implementation issues. I think, as others have pointed out, that there are kernel settings for TCP that probably need adjustment, but they are not exposed in ROS so you cannot. In linux:
Command
semmsl, semmns, semopm, and semmni 	# /sbin/sysctl -a | grep sem

This command displays the value of the semaphore parameters in the order listed.
shmall, shmmax, and shmmni 	# /sbin/sysctl -a | grep shm

This command displays the details of the shared memory segment sizes.
file-max 	# /sbin/sysctl -a | grep file-max

This command displays the maximum number of file handles.
ip_local_port_range 	# /sbin/sysctl -a | grep ip_local_port_range

This command displays a range of port numbers.
rmem_default 	# /sbin/sysctl -a | grep rmem_default

Others.
rmem_max 	# /sbin/sysctl -a | grep rmem_max
wmem_default 	# /sbin/sysctl -a | grep wmem_default
wmem_max 	# /sbin/sysctl -a | grep wmem_max
tcp_wmem 	# /sbin/sysctl -a | grep tcp_wmem
tcp_rmem 	# /sbin/sysctl -a | grep tcp_rmem
Can MT expose control for kernel TCP settings in ROS (like above)?

I would not be surprised if there are also PCI bus timing issues at play here. Under linux, up until 2.6.34 and 3.7.1 (most recent kernel updates) there was a problem with enumerating the PCI bus for the AR9xx chips on AR7xxx cpus. If you had more than one radio module on the RB433, the ath9k driver would only find the first one. They have now fixed this in both Kernel and in the driver. ROS did not appear to have this problem, but who knows, maybe it was there but not fully broken like in linux. If you have PCI bus timing problems, and you have TCP sync problems in your wireless driver, and you have bad TCP settings in the kernel, then how can you expect this to work even as well as it does.

This all said, we have another twist to add to this. We have a point to point to multipoint configuration. We are using NV2 with WDS, 2x2, 20/40MHz, AES, ROS5.21. When we first set up the links, it worked fine for about a week, then all of a sudden the links went down. Boards are RB433AH and radios are R52Hn. Antennas are cross pol panel and cross pol sector.

The common problem is that chain 1 went from signal of -62db to -89db on three different routers. Almost all at the same time. The channels on the radios are all different and widely separated. There is no noise, no other traffic. Spectral scan finds nothing in the area. These are remote from anything.

If we downgrade from 5.21 to 4.17, then the chains appear to come back into balance, but the link will not come up.

If we swap out the RB433AH for a RB435G, then the links come back and chain 1 and chain 0 are both -63dB. Same exact configuration, same power supply, same antennas, same radio modules, same ROS version.

Is there something wrong with the PCI bridge or CPLD on the RB433AH? Is there something about having two or more AR9xxx radios on the same bus? At first we thought it was power, now I think it is a driver or chip issue. Noticed that the CPLD on RB433AH is the Xilinx XC9572xl, but this chip is gone on RB433GL and newer boards. I will have to look at the RB435G to see which chip it has.

Seems like a pretty deep technical issue that MT has got to apply some resources to.
WOOF BANG!
 
ffernandes
Member Candidate
Member Candidate
Posts: 146
Joined: Mon Jun 23, 2008 11:20 pm

Re: nv2 multilink problem

Wed Jan 16, 2013 6:32 pm

any news about this finding?!?
 
denis
Frequent Visitor
Frequent Visitor
Posts: 71
Joined: Tue Sep 20, 2005 9:27 pm
Location: Serbia

Re: nv2 multilink problem

Wed Jan 16, 2013 9:25 pm

Nope. Just high hopes ...
 
sonny
Member Candidate
Member Candidate
Posts: 208
Joined: Fri Jan 28, 2005 5:14 pm
Location: Germany
Contact:

Re: nv2 multilink problem

Thu Jan 17, 2013 2:27 pm

Hi Uldis,
it seems the problem exists mostly with mibsbe devices and ROS 5.18 up, not mainly a wireless problem.
Please reconfigure your test lab like tihs:

RB1000---ethernet---RB711UA-5HnD/\/\bridged-wireless2x2(20mhz)/\/\RB711-5HnD---ethernet---routed-RB433/or/RB2011----ethernet---RB711UA-5HnD/\/\bridged-wireless2x2(20mhz)/\/\RB711-5HnD---ethernet---RB800/or/RB1000

Better you add a third wireless link then you will see the drop of tcp bandwith.

Then test each link separate with TCP single stream and between the far ends.

We have problems with a location three wireless Hops away connected through bridged wireless links and between the links routed mibsbe devices.
Each single link will do 30mbit TCP singel stream, but from the far end you can get only 3-6mbit.

Just made a lab test with 60db attenuation between the wireless boards (signal approx -50) with 2x2 Mimo and 20mhz channel width - data rates 130mbps. Wireless modes ap-bridge on the AP and station-bridge on the Client. On AP and on the Client ethernet interface is bridged with wireless interface.

RB1000---ethernet---RB711UA-5HnD/\/\wireless2x2(20mhz)/\/\RB711-5HnD---ethernet---RB800

Wireless board running v5.21 with RouterBoard firmware v3.0 and RB800/RB1000 RouterOS v6.0rc2.

Bandwidth test results:
UDP test between the RB1000<->RB800 is approx 97Mbps
TCP test 1 connection 88-92Mbps
TCP test 20 connections 90-93Mbps
Karl Sonnleitner
Senior Wireless-Expert
Restlesspowerbox - managed powersupply for Routerboards
 
wpeople
Member
Member
Posts: 352
Joined: Sat May 26, 2007 6:36 pm

Re: nv2 multilink problem

Thu Jan 17, 2013 8:53 pm

hmm. hoping to be solved. By the way, what about running UDP based tunnels between the internet uplink point and last mile?
Maybe it eliminates TCP timing issues?

maybe changing from:
Internet->AP -(wireless)- WDS Station - Switch SiteA- AP -(wireless)-WDS Station - Switch SiteB - AP -(wireless)-Station bridge-Customer Router
to
Internet->AP -(wireless)- WDS Station - Switch SiteA- StationWDS -(wireless)-WDS Bridge - Switch SiteB - AP -(wireless)-Station bridge-Customer Router

(so instead of AP-client - AP client to AP-client-client-AP))

Anyone tried that scenario?

Edit: i've tried to change ap/staWDS to staWDS/AP on 2nd wireless link - not helped.
What is strange: Internet->node2 ~20mbps while measuring the opposite direction is double...
 
User avatar
ringlet
newbie
Topic Author
Posts: 35
Joined: Sun Mar 07, 2010 9:24 pm
Location: Gdynia, Poland
Contact:

Re: nv2 multilink problem

Fri Jan 18, 2013 3:23 am

does anyone try to increase MTU value on every device in the row?
I'm not sure but i have suspicion that there can be a problem

http://wiki.mikrotik.com/wiki/Maximum_T ... 2FL2.5_MTU
http://forum.ubnt.com/showthread.php?t=51867
https://forum.ubnt.com/showthread.php?t=51132&page=3

i can't find if normal bridge mode increase somehow packet size and if, how much?

in next week i will try to increase mtu size over problematic multihop-link in the night and we will see
...
 
sonny
Member Candidate
Member Candidate
Posts: 208
Joined: Fri Jan 28, 2005 5:14 pm
Location: Germany
Contact:

Re: nv2 multilink problem

Fri Jan 18, 2013 10:12 am

We have not solved the problem yet.
But, we are now running the midle wireless link with nstreme and bridged the last mile (PtMP customer sector) in a eoip tunnel to the first router in the row.

customer SXT //-wireless NV2 MIMO-// (PtMP) RB433 rotued (bridged wlan to eoip1) ->

SXT (station bridged) //-wirless nstreme MIMO-// SXT(AP bridged) ->

RB2011 (routed) eoip in Bridge with customer net on Bridge ->

RB711 (stationbridge) //-wireless NV2 MIMO-// RB411AH (AP bridged) ->

to uplink

this config brought us from 2-3mbit/s to 15-18 MBit per single stream at customers SXts and of course better ping times :D
Karl Sonnleitner
Senior Wireless-Expert
Restlesspowerbox - managed powersupply for Routerboards
 
wpeople
Member
Member
Posts: 352
Joined: Sat May 26, 2007 6:36 pm

Re: nv2 multilink problem

Fri Jan 18, 2013 10:37 am

does anyone try to increase MTU value on every device in the row?
I'm not sure but i have suspicion that there can be a problem

in next week i will try to increase mtu size over problematic multihop-link in the night and we will see
...
Hm.
1) I don't think the packet size would change anything, since standard (1500byte) ping with DoNotFragment goes thru well
2) unless you are using MPLS/VPLS (or PPPoE) everything works well with 1500byte packets

What is strange to me is downlink is slower than uplink (internet feed router to CPE).
 
User avatar
interpoint
Frequent Visitor
Frequent Visitor
Posts: 91
Joined: Wed Jun 06, 2007 3:42 pm
Location: Ireland
Contact:

Re: nv2 multilink problem

Wed Jan 23, 2013 10:38 am

I am wondering if this is a buffer overrun issue. Buffer fills before it can empty to the next link so it drops packets and they need to be resent. This does not happen when each segment is tested in isolation but happens when multi segments come together in a row.

Try using VLAN switch chip bandwidth control feature.
This is not available on all router boards. RB1100AH has it for example.
Then set the vlan switch chip port speed to say 20 MB on a router in the middle of the multihop segment.
See if this affects the multihop end to end speed.

VLAN chip based speed control operates much lower than HTB. Much closer to the pulsing speeds of the Ethernet port.
So in theory, traffic should be slowed and upstream wireless buffers should not overfill and packets get dropped.

This experiment is not a solution but may show the cause of the issue.
Shane Hartigan
Director - Integrated Media Solutions
MikroTik Certified Trainer
MTCNA, MTCRE, MTCWE, MTCINE, MTCTCE, MTCIPv6E
 
rado3105
Member
Member
Posts: 480
Joined: Sat Jan 12, 2008 11:45 pm

Re: nv2 multilink problem

Wed Jan 23, 2013 11:24 pm

Yes I am also thinking this is bufferbloat problem:
http://twit.tv/sn/359
http://queue.acm.org/detail.cfm?id=2209336
 
wpeople
Member
Member
Posts: 352
Joined: Sat May 26, 2007 6:36 pm

Re: nv2 multilink problem

Sun Feb 24, 2013 12:19 pm

Please see the following thread:

http://forum.mikrotik.com/viewtopic.php?f=7&t=70130

When gigE comes in (especially with NV2) it degrades the effectiveness off fastE devices.
Measuring from/to full 100mbps network is OK. but if the traffic is starting from gigE endpoint (measurement or real life traffic), than speed is only of fraction than expected.

changing all the devices to feed with gigE, solves the problem.

Seems to have backpressure(?) problem? Or gigE want the ACKs faster??
 
czupakab
just joined
Posts: 1
Joined: Sun Mar 10, 2013 3:46 pm

Re: nv2 multilink problem

Sun Mar 10, 2013 3:48 pm

Any news. Any hints ?
 
wpeople
Member
Member
Posts: 352
Joined: Sat May 26, 2007 6:36 pm

Re: nv2 multilink problem

Mon Mar 11, 2013 6:02 pm

czupakab:
see the above thread: the solution (for me) is to replace the wired path from fast-ethernet to GigE if your internet router is with GigE.
 
nemirko
just joined
Posts: 19
Joined: Sat Dec 01, 2012 10:52 pm

Re: nv2 multilink problem

Mon Oct 07, 2013 2:40 pm

is this problem with multihop for nv2 and nstream solved in version 6.4??
 
karlos
Frequent Visitor
Frequent Visitor
Posts: 81
Joined: Sun Jan 16, 2011 10:46 am

Re: nv2 multilink problem

Tue Oct 08, 2013 2:18 pm

is this problem with multihop for nv2 and nstream solved in version 6.4??
*) wireless - improved 802.11n wireless retransmission (doesn't effect nstreme/nv2)
 
braidiano
Frequent Visitor
Frequent Visitor
Posts: 85
Joined: Sat Dec 11, 2010 1:29 am

Re: nv2 multilink problem

Fri Jan 17, 2014 5:03 pm

Hello,

any news from mikrotik? Do you have a workaround to fix or make the thinghs better?
 
plisken
Forum Guru
Forum Guru
Posts: 2421
Joined: Sun May 15, 2011 12:24 am
Location: Belgium
Contact:

Re: nv2 multilink problem

Sun Jan 19, 2014 1:25 pm

Try this configuration en let know of this works

http://www.wirelessinfo.be/index.php/mi ... /omnitikpp

Other device but is the same configuration.

Goodluck
 
plisken
Forum Guru
Forum Guru
Posts: 2421
Joined: Sun May 15, 2011 12:24 am
Location: Belgium
Contact:

Re: nv2 multilink problem

Sun Jan 19, 2014 1:58 pm

I have this try and working fine
 
Cetalfio
Member Candidate
Member Candidate
Posts: 224
Joined: Sat Sep 20, 2008 6:19 pm
Location: Italy

Re: nv2 multilink problem

Mon Jan 20, 2014 4:21 pm

This thread is still valid, the problems have been solved with the 6.7?

cetalfio
 
braidiano
Frequent Visitor
Frequent Visitor
Posts: 85
Joined: Sat Dec 11, 2010 1:29 am

Re: nv2 multilink problem

Tue Jan 21, 2014 12:58 am

I tried the configuration from plisken, but did not work for me :-( any other advice or solution from mikrotik?

Inviato dal mio M-MP875S2 utilizzando Tapatalk
 
plisken
Forum Guru
Forum Guru
Posts: 2421
Joined: Sun May 15, 2011 12:24 am
Location: Belgium
Contact:

Re: nv2 multilink problem

Tue Jan 21, 2014 10:45 am

What's on to read logfile
add to logfile "wireless-debug

click on "system"
click on "logging"
add wireless debug
See printscreens a made
You do not have the required permissions to view the files attached to this post.
 
User avatar
sten
Forum Veteran
Forum Veteran
Posts: 920
Joined: Tue Jun 01, 2004 12:10 pm

Re: nv2 multilink problem

Fri Feb 21, 2014 12:22 pm

Seems to me that you guys are running into issues with TCP. Are you perhaps running Nv2 aggregation AND 802.11n aggregation?
Move along. Nothing to see here.
 
digicomtech
Frequent Visitor
Frequent Visitor
Posts: 75
Joined: Fri Apr 20, 2007 5:03 pm
Location: Alma, Qc, Canada
Contact:

Re: nv2 multilink problem

Fri Feb 21, 2014 6:53 pm

Just to confirm the theory of bufferbloat, we have this setup:

Internet 300 Mbps FD <-some firewall-> RB1100AH <-> RB450G <-> RB711GA <---200 Mbps HD wireless link ---> RB912UAG <---> RB1100AHx2

Doing a BTest between RB1100AH and RB1100x2 with real traffic traveling in this link:

TCP send 1 connection: Hardly 13 Mbps 10 sec Average

After I have read this forum and a part of the bufferbloat theory, I take a look of the queue on the RB1100AH and notice that the only queue we got was the default hardware-queue-only, so no software queue at all. I decide to put a pfifo 100 packets queue on the interface facing this wireless link and here is the result of the Btest:

TCP send 1 connection: 40 Mbps 10 sec Average

Looking at the real traffic on the interface it reach over 100 Mbps; so the result of the Btest is more acceptable than it was before.

I will do more tests over mutiple NV2 hops and let you know.

Regards,
Michael
 
braidiano
Frequent Visitor
Frequent Visitor
Posts: 85
Joined: Sat Dec 11, 2010 1:29 am

Re: nv2 multilink problem

Sat Feb 22, 2014 12:44 am

I setup the pfifo queue to the 750's ethernet connected to the 711, but it doesnt help for me

Inviato dal mio M-MP875S2 utilizzando Tapatalk
 
digicomtech
Frequent Visitor
Frequent Visitor
Posts: 75
Joined: Fri Apr 20, 2007 5:03 pm
Location: Alma, Qc, Canada
Contact:

Re: nv2 multilink problem

Mon Feb 24, 2014 11:19 pm

Seems to me that you guys are running into issues with TCP. Are you perhaps running Nv2 aggregation AND 802.11n aggregation?
What do you mean by NV2 aggregation ?
Does the 802.11n aggregation setting apply to the NV2 protocol also ?
As I can read NV2 do packet aggregation and this aggregation is not configurable from the user interface...

In our setup we haven't modify the default 802.11n aggregation settings. So only the priority 0 use aggregation. I guess if no mangle rules is set on the device then every traffic is using aggregation :P

Am I right ?

Regards,
Michael
 
digicomtech
Frequent Visitor
Frequent Visitor
Posts: 75
Joined: Fri Apr 20, 2007 5:03 pm
Location: Alma, Qc, Canada
Contact:

Re: nv2 multilink problem

Mon Feb 24, 2014 11:21 pm

I setup the pfifo queue to the 750's ethernet connected to the 711, but it doesnt help for me

Inviato dal mio M-MP875S2 utilizzando Tapatalk

Can you describe your setup and the configuration you use on wireless link ?

Regards,
Michael
 
User avatar
sten
Forum Veteran
Forum Veteran
Posts: 920
Joined: Tue Jun 01, 2004 12:10 pm

Re: nv2 multilink problem

Tue Feb 25, 2014 6:15 am

802.11n aggregation is in addition to nv2 aggregation. At least it used to be (it's been a year since i last tweaked a wireless link).
Move along. Nothing to see here.
 
ste
Forum Guru
Forum Guru
Posts: 1805
Joined: Sun Feb 13, 2005 11:21 pm

Re: nv2 multilink problem

Tue Feb 25, 2014 7:54 am

802.11n aggregation is in addition to nv2 aggregation. At least it used to be (it's been a year since i last tweaked a wireless link).
How will this work? When nv2 aggregates small packets to a bigger one how can it be further aggregated?
 
User avatar
bclewl1ns
Frequent Visitor
Frequent Visitor
Posts: 60
Joined: Sat Jul 06, 2013 7:38 am
Contact:

Re: nv2 multilink problem

Wed Apr 29, 2015 12:48 am

Still having issues on this with V6.27 I have links that used to run at 75 meg not barely getting 30. But on UDP i can test at 100. Is their any way to fine tune NV2 as i can not find anything.
Brett

What is understood need not be discussed.
 
rado3105
Member
Member
Posts: 480
Joined: Sat Jan 12, 2008 11:45 pm

Re: nv2 multilink problem

Mon May 04, 2015 5:33 pm

Still having issues on this with V6.27 I have links that used to run at 75 meg not barely getting 30. But on UDP i can test at 100. Is their any way to fine tune NV2 as i can not find anything.
Try another firmware, lik 6.19 and also with wirelles-fp enabled..and give us know....
 
User avatar
Giepie
Member
Member
Posts: 431
Joined: Mon Sep 13, 2004 12:33 pm
Location: Western Cape, South Africa
Contact:

Re: nv2 multilink problem

Thu Jul 09, 2015 9:40 pm

Hi guys

Hopefully my post would shed some more light on the problem. I have a similar setup as most of you, but with the exception of using Ubiquity radio's and Mikrotik Routers.

Here is my scenario:

AAA --> BBB --> CCC

AAA is connected to BBB using a PtP Ubiquity link (2x Powerbeam 25dBi's), and on its own I get near 100mbps in TCP or UDP.

BBB is connected to CCC also using a PtP Ubiquity link (2x 30dBi dishes with Rockets), and on its own I get close to 100mbps in TCP or UDP.

But when I test from AAA to CCC, I get close to 100mbps UDP, and only 10-20mbps TCP throughput.

Since this is a very important link, I can't play with it too much. Currently my consideration is to put those 4 radios on the same subnet (ie, use the switch chip on the Mikrotik at BBB). Even if this works, it is not the correct solution I was looking for, but it would at least make my one backbone link much better (if it works!)


If I have time, I would like to test the following scenario (unless someone has already tried it):

PC1 -ethernet-> MT-AAA -ethernet-> MT-BBB -ethernet-> MT-CCC -ethernet-> PC2

In this test you will exclude wireless from the equasion completely and only test raw Routerboard performance. I'll set it up as a fully routed network, and possibly compare it to just bridging (using switch chip) all interfaces together and using a single IP subnet.

Has anyone tried my above scenario, and what was your results?

Thanks guys! G
The only thing Mikrotik lack, is to send power to your High Sites wirelessly.....
 
User avatar
PCaddict69
just joined
Posts: 17
Joined: Fri Jun 29, 2007 6:40 pm

Re: nv2 multilink problem

Tue Sep 01, 2015 12:43 am

HI , i just start test Cambium GPS sync AP in the 2.4ghz and i got Strange problem when i hit the nv2... 3 to 4 mbps througput in PPPOE... But if i connect to pppoe from the 750UP at the base station they go well!. If i deactivated the NV2 and go to 802.11 of the backhaul from the Base Station to the next Hop Things go WELL!. But as soon i hit a link that have NV2 on the way the tcp became poor as hell. The network is fully routed with mpls and the pppoe go Inside a vpls tunnel to my concentrator. I test on layer 3 too.

From my computer to the Cambium SM to the Cambium AP to the 750UP. No problem

But as soon i hit the backhaul to the next hop the tcp drop to 3-4 mbps... here the design of the network of this hop

RB2011 ------- RB912AP ------------- RB912STA-----RB750UP-----EPMP1000APGPS ----- EPMP1000SM ----- My computer

2011 TO 912 /30 Route RB912AP to STA /30 route RB912 to 750 and EPMP1000 Switching

As long they are no NV2 on the way i got a solid 30Mbps (The max i got from the EPMP in 10Mhz 2.4)

I can make my bandwidth test to the 912STA with solid 30Mbps and to the RB2011 if they are on 802.11 but as soon i kick the nv2 i got 3-4 mbps same things if i go beyond the rb2011 is routed to another rb912 to my main tower that are on NV2.
 
User avatar
bclewl1ns
Frequent Visitor
Frequent Visitor
Posts: 60
Joined: Sat Jul 06, 2013 7:38 am
Contact:

Re: nv2 multilink problem

Thu Sep 03, 2015 9:18 pm

So after multiple designed I have found a common problem. Any link i have running in NV2 can not go across another NV2 Link. I actually went out and purchased a Ubiquity Airfiber 24 HD to replace the backhaul from one of our locations. I then backed it up with a NetMetal PTP running NV2. The clients are connected VIA NV2. When we did speed tests form a client location through the Airfiber backhaul we got 60Mbps Download TCP and 100Mpbs Download UDP. (Still don't see why this is happening) However, When we failed over to the Netmetal PTP. Our download BW went down to 20Mbps TCP but we still received 100 Mbps upload UDP.

I am scratching my head going WTH. At this point we are starting to plan a completed migration from all mikrotik products for future growth on the Multipoint Side. and Immediately moving from Mikrotik for all current PTP links.

I don't want to do this HOWEVER. We replaced the Netmetal PTP link with a Ubiquity Airfiber 5. (Same Antenna same alignment, same freq) and We got 60 Mpbs TCP.

Please Mikrotik I have been using your products for over 6 years. PLEASE ADDRESS this as i don't want to have to move EVERYTHING to Ubiquity.

We also Used Wireless-fp and tested with firmwares.

6.18
6.19
6.23
6.27
6.29
and finally 6.31 (NO SIGNIFICANT DIFFERENCE)
Brett

What is understood need not be discussed.
 
User avatar
PCaddict69
just joined
Posts: 17
Joined: Fri Jun 29, 2007 6:40 pm

Re: nv2 multilink problem

Sat Sep 05, 2015 5:42 pm

bclewl1ns we are on the same boat. I willing to pay if someone can fix my problem. So if there anybody have a pratical solutions before i start to phase out my mikrotik wireless backhaul let me know... I willing to pay!
 
ste
Forum Guru
Forum Guru
Posts: 1805
Joined: Sun Feb 13, 2005 11:21 pm

Re: nv2 multilink problem

Sat Sep 05, 2015 9:00 pm

bclewl1ns we are on the same boat. I willing to pay if someone can fix my problem. So if there anybody have a pratical solutions before i start to phase out my mikrotik wireless backhaul let me know... I willing to pay!
This is a long time problem. We migrate to licensed or Mimosa B5 on most backhaul. Only smaller sites stay on MT-Backhaul. With newer Versions it got better but we still see low tcp throughput.
 
User avatar
bclewl1ns
Frequent Visitor
Frequent Visitor
Posts: 60
Joined: Sat Jul 06, 2013 7:38 am
Contact:

Re: nv2 multilink problem

Sat Sep 05, 2015 11:48 pm

I MAY HAVE SOME HELP.

The wireless-fp package is now the legacy package for Mikrotik. I moved to the new wireless-cm package and i have to admit that the TCP throughput is much better. The caveat is however you have to be using router os versions 6.33rc3 minimum.

As far as the last post I have also used mimosa. I have now taken all of them out of my network as they work very poorly is even mildly noisy environments and if the radio of any reason loses GPS sync it resets the link. I lost a customer over that &*%*#@* radio. I only have 1 left in production and that is a dedicated POINT TO POINT link in the middle of a farm with nothing around it.

So in short Try the wireless-cm package in ROS 6.33rc3 or higher. It does improve things quite noticeably.
Brett

What is understood need not be discussed.
 
ibm
Member
Member
Posts: 301
Joined: Mon May 12, 2014 5:16 pm

Re: nv2 multilink problem

Sun Sep 06, 2015 12:38 am


The wireless-fp package is now the legacy package for Mikrotik. I moved to the new wireless-cm package and i have to admit that the TCP throughput is much better. The caveat is however you have to be using router os versions 6.33rc3 minimum.
Are you sure about this?
Is the wireless cm not only related to capsman function?
 
ste
Forum Guru
Forum Guru
Posts: 1805
Joined: Sun Feb 13, 2005 11:21 pm

Re: nv2 multilink problem

Sun Sep 06, 2015 12:41 am

I MAY HAVE SOME HELP.

The wireless-fp package is now the legacy package for Mikrotik. I moved to the new wireless-cm package and i have to admit that the TCP throughput is much better. The caveat is however you have to be using router os versions 6.33rc3 minimum
There is a changelog entry claiming increased tcp speed with .ac chipset. So MT seems to see the problem now and takes steps to improve things.
As far as the last post I have also used mimosa. I have now taken all of them out of my network as they work very poorly is even mildly noisy environments and if the radio of any reason loses GPS sync it resets the link. I lost a customer over that &*%*#@* radio. I only have 1 left in production and that is a dedicated POINT TO POINT link in the middle of a farm with nothing around it.
We've great success with this radios. But you're right they are sensitive to self interference with (foreign) radios not in sync with them. You've to install them with >3m separation or with enough frequency space between. Newest release improves this.
So in short Try the wireless-cm package in ROS 6.33rc3 or higher. It does improve things quite noticeably.
 
User avatar
bclewl1ns
Frequent Visitor
Frequent Visitor
Posts: 60
Joined: Sat Jul 06, 2013 7:38 am
Contact:

Re: nv2 multilink problem

Sun Sep 06, 2015 7:17 am


The wireless-fp package is now the legacy package for Mikrotik. I moved to the new wireless-cm package and i have to admit that the TCP throughput is much better. The caveat is however you have to be using router os versions 6.33rc3 minimum.
Are you sure about this?
Is the wireless cm not only related to capsman function?

Yes. her is the info in another post. http://forum.mikrotik.com/viewtopic.php?t=99017
Brett

What is understood need not be discussed.
 
User avatar
bclewl1ns
Frequent Visitor
Frequent Visitor
Posts: 60
Joined: Sat Jul 06, 2013 7:38 am
Contact:

Re: nv2 multilink problem

Sun Sep 06, 2015 7:21 am

I MAY HAVE SOME HELP.

The wireless-fp package is now the legacy package for Mikrotik. I moved to the new wireless-cm package and i have to admit that the TCP throughput is much better. The caveat is however you have to be using router os versions 6.33rc3 minimum
There is a changelog entry claiming increased tcp speed with .ac chipset. So MT seems to see the problem now and takes steps to improve things.
As far as the last post I have also used mimosa. I have now taken all of them out of my network as they work very poorly is even mildly noisy environments and if the radio of any reason loses GPS sync it resets the link. I lost a customer over that &*%*#@* radio. I only have 1 left in production and that is a dedicated POINT TO POINT link in the middle of a farm with nothing around it.
We've great success with this radios. But you're right they are sensitive to self interference with (foreign) radios not in sync with them. You've to install them with >3m separation or with enough frequency space between. Newest release improves this.
So in short Try the wireless-cm package in ROS 6.33rc3 or higher. It does improve things quite noticeably.

Thanks for the info. We had GPS Sync problem with Firmware 1.11 and Stability problems with 1.2 I see now they have release 1.2.2 I am hoping we can start using them again
Brett

What is understood need not be discussed.
 
User avatar
PCaddict69
just joined
Posts: 17
Joined: Fri Jun 29, 2007 6:40 pm

Re: nv2 multilink problem

Thu Sep 17, 2015 5:37 pm

Hi,
First sorry for my bad english...
I make a LOOOOOT of test in the 2 last week. But i think i got a hint. TCP Window Scaling... I have a network i mostly use only for test with client aware of and i give them a rebate.... On the last hop i have a Rocket Titanium M2 for ap to client(on airmax) and a Cambium AP EPMP 1000 2 Ghz GPS on it.Each client have the cambium SM (connecterized and non connecterize) and a UBNT SM (I have all ubnt product distribute from one client to other(dish , loco , nano , rocket , pico , bullet , and so on...)Each client get internet access from PPPOE on a 750UP so i can switch on the fly the antenna they use. A RB750UP Switch at the bottom and The backhaul to the next hop was two MTK RB912 with grid dish antenna. Later change with two Dynadish AC. The network is fully routed with MPLS and vpls tunnel to my concentrator.

With the time a discover they are no good lab test... The real test is on the field...

First to get the maximum speed on the RB912 i change the tx power to 17 dbm. They create a looot of noise at normal tx power.

TCP speed on pppoe are perfect at the 750UP before enter to client AP.

But from the client the speed are around 3 to 7 mbps.... from speedtest.net and my local speedtest page sit next to the pppoe concentrator a CCR1016.

If i deactivated nv2 the client can get full tcp speed to the next hop.

So i change for 2 dynadish. Same results.

Soon you add latency at the last hop you get into trouble... So with a backhaul on NV2 the cambium perform the worst (because of the latency add by the gps sync). So the only thing that pop in my head is tcp window scaling... Im not a tcp specialist. Maybe im wrong but what i see point me to this....
 
digicomtech
Frequent Visitor
Frequent Visitor
Posts: 75
Joined: Fri Apr 20, 2007 5:03 pm
Location: Alma, Qc, Canada
Contact:

Re: nv2 multilink problem

Wed May 11, 2016 6:51 pm

Hi guys,
we had a lot of link PTP and even PTMP with NV2 on it.
Best solution for us is the new wireless-rep package 6.35.1.
For exemple:

All RBs in routed mode, no MPLS, just OSPF.

Core Head
RB1100AHx2 -- RB1100 -- RB911G-5HPacD ---- Wireless-rep NV2 link --- RB911G-5HPacD --- RB1100AHx2 --- RB911G-5HPacD ---- Wireless-rep NV2 link --- RB911G-5HPacD -- RB450G

From the RB450G we manage a BTest TCP 4 connections with the RB1100AHx2 at the core head:

Avg: 102 Mbps
This is a production link so there is always at least 30 to 40 Mbps of real traffic.

On every link with good signal / CCQ we had put the wireless-rep package, we got a lot of improvement with TCP throughput. wireless-cm2 was good also, but we got a lot of WDS interface and NV2 fixed for WDS is only available in the rep package.

As other as suggest, it is clearly a TCP window scaling problem.
Reduce latency on every link got a direct impact on TCP troughput.
When possible, use NStreme as latency is a lot better, specially with PTP link.
However, I had not tested NStreme in wireless package rep and others.

Hope it will help some of you.
Regards,
Michael Plourde
Digicom

Who is online

Users browsing this forum: No registered users and 6 guests