Page 1 of 1

unstable ping with nstream

Posted: Sun Aug 03, 2008 9:56 am
by ropebih
does someone know why is ping unstable with Nstream? I have ptp link, on both side is PC with installed mikrotik. MT1 is version 3.9 AP bridge and MT2 is version 3.11 station WDS.

with activated Nstream troughout empty link ping is unstable, it goes over 20 ms. But, without nstream ping is stable, <1 ms.

With nstream:
10.10.60.9 64 byte ping: ttl=64 time<1 ms
10.10.60.9 64 byte ping: ttl=64 time=2 ms
10.10.60.9 64 byte ping: ttl=64 time=1 ms
10.10.60.9 64 byte ping: ttl=64 time=1 ms
10.10.60.9 64 byte ping: ttl=64 time=1 ms
10.10.60.9 64 byte ping: ttl=64 time=17 ms
10.10.60.9 64 byte ping: ttl=64 time=3 ms
10.10.60.9 64 byte ping: ttl=64 time=1 ms
10.10.60.9 64 byte ping: ttl=64 time=5 ms
10.10.60.9 64 byte ping: ttl=64 time=1 ms
10.10.60.9 64 byte ping: ttl=64 time=2 ms
10.10.60.9 64 byte ping: ttl=64 time=1 ms
10.10.60.9 64 byte ping: ttl=64 time<1 ms
10.10.60.9 64 byte ping: ttl=64 time=1 ms
10.10.60.9 64 byte ping: ttl=64 time=2 ms
10.10.60.9 64 byte ping: ttl=64 time=1 ms
10.10.60.9 64 byte ping: ttl=64 time=21 ms



without nstream:
10.10.60.9 64 byte ping: ttl=64 time<1 ms
10.10.60.9 64 byte ping: ttl=64 time<1 ms
10.10.60.9 64 byte ping: ttl=64 time<1 ms
10.10.60.9 64 byte ping: ttl=64 time<1 ms
10.10.60.9 64 byte ping: ttl=64 time<1 ms
10.10.60.9 64 byte ping: ttl=64 time<1 ms
10.10.60.9 64 byte ping: ttl=64 time<1 ms
10.10.60.9 64 byte ping: ttl=64 time=1 ms
10.10.60.9 64 byte ping: ttl=64 time<1 ms
10.10.60.9 64 byte ping: ttl=64 time<1 ms
10.10.60.9 64 byte ping: ttl=64 time<1 ms
10.10.60.9 64 byte ping: ttl=64 time<1 ms

Re: unstable ping with nstream

Posted: Sun Aug 03, 2008 5:09 pm
by jwcn
That is normal ping times (actually quite good) for an N stream link with little or no load.

Re: unstable ping with nstream

Posted: Sun Aug 03, 2008 8:52 pm
by JJCinAZ
Try same test with and without NStream AND running a large amount of traffic through the link. Without NStream you should see wild latency values (1ms to 1000ms). With Nstream, the values during load should be much more consistent.

Re: unstable ping with nstream

Posted: Wed Aug 06, 2008 7:15 pm
by ilius168
Same here
i have got a few RB411A connected using CM9 (and many other) cards.

It seems like the latency gone unstable with nstreme enabled, from 1 digit to sometimes 3 digits, while it's stable at 1ms without. Though throughput is much higher with nstreme.

Is there any setting i need with nstreme?

Any comment would be appreciated. Thanks!

Ping with nstreme
172.168.77.32 64 byte ping: ttl=64 time=1 ms
172.168.77.32 64 byte ping: ttl=64 time=19 ms
172.168.77.32 64 byte ping: ttl=64 time=1 ms
172.168.77.32 64 byte ping: ttl=64 time=2 ms
172.168.77.32 64 byte ping: ttl=64 time=5 ms
172.168.77.32 64 byte ping: ttl=64 time=2 ms
172.168.77.32 64 byte ping: ttl=64 time=189 ms
172.168.77.32 64 byte ping: ttl=64 time=1 ms
172.168.77.32 64 byte ping: ttl=64 time=9 ms
172.168.77.32 64 byte ping: ttl=64 time=1 ms
172.168.77.32 64 byte ping: ttl=64 time=18 ms
Ping without nstreme
172.168.77.32 64 byte ping: ttl=64 time=1 ms
172.168.77.32 64 byte ping: ttl=64 time=1 ms
172.168.77.32 64 byte ping: ttl=64 time=1 ms
172.168.77.32 64 byte ping: ttl=64 time=2 ms
172.168.77.32 64 byte ping: ttl=64 time=1 ms
172.168.77.32 64 byte ping: ttl=64 time=1 ms
172.168.77.32 64 byte ping: ttl=64 time=1 ms
172.168.77.32 64 byte ping: ttl=64 time=1 ms
172.168.77.32 64 byte ping: ttl=64 time=1 ms

My config looks like this:
[van@RAH-1] /interface wireless> pri
Flags: X - disabled, R - running 
 0  R name="wlan1" mtu=1500 mac-address=00:0B:6B:56:8B:13 
      arp=enabled interface-type=Atheros AR5213 mode=ap-bridge
      ssid="IRAH" frequency=2404 band=2.4ghz-g-turbo 
      scan-list=2400-2539 antenna-mode=ant-a wds-mode=dynamic 
      wds-default-bridge=bridge1 wds-ignore-ssid=no 
      default-authentication=yes default-forwarding=yes 
      default-ap-tx-limit=0 default-client-tx-limit=0 hide-ssid=no 
      security-profile=default compression=no 

[van@RAH-1] /interface wireless nstreme> pri
 0 name="wlan1" enable-nstreme=yes enable-polling=yes 
   disable-csma=no framer-policy=none framer-limit=3200 
[van@RAH-1] /interface wireless nstreme> /sys resource pri
                   uptime: 2h3m39s
                  version: "3.10"
              free-memory: 52260kB
             total-memory: 62616kB
                      cpu: "MIPS 24K V7.4"
                cpu-count: 1
            cpu-frequency: 300MHz
                 cpu-load: 14
           free-hdd-space: 34512kB
          total-hdd-space: 61440kB
  write-sect-since-reboot: 247
         write-sect-total: 83495
               bad-blocks: 0
        architecture-name: "mipsbe"
               board-name: "RB411"

Re: unstable ping with nstream

Posted: Wed Aug 06, 2008 8:00 pm
by jwcn
Question has already been asked an answered.

Re: unstable ping with nstream

Posted: Wed Aug 06, 2008 8:06 pm
by ropebih
Question has already been asked an answered.
anwer is not real. There is somany links with nstream and version 2.9 and ping is very stable (1-5 ms on 20 mbita troughpt). But, this is my firt link with version 3 and nstream and ping is very unstable on empty link.

Re: unstable ping with nstream

Posted: Thu Aug 07, 2008 5:03 am
by jwcn
Do a search on N stream. It has always been this way. Links with little to no load have erratic latency.

Re: unstable ping with nstream

Posted: Thu Aug 07, 2008 7:41 am
by RK
Do a search on N stream. It has always been this way. Links with little to no load have erratic latency.
While you are it, read up on what nstream actually does and how it does it. Everything will become clear then.

Re: unstable ping with nstream

Posted: Thu Aug 07, 2008 12:16 pm
by normis
BTW JFYI:

x-treme --> n-streme

(as far as the name goes, nothing to do with topic)

Re: unstable ping with nstream

Posted: Fri Sep 05, 2008 11:10 am
by ilius168
so is there anyway to overcome the latency problem when using nstreme?

cos' we have users that is all the time looking at the ping time and started complaining, but in fact the ping time doesn't always effect internet (cmiiw).

Re: unstable ping with nstream

Posted: Fri Sep 05, 2008 3:31 pm
by Egate
Nstream needs a load to be stable. If low speed over nstream, very erratic. Tel client to ping while downloading or get more users.

Re: unstable ping with nstream

Posted: Mon Sep 08, 2008 9:26 am
by macgaiver
Nstreme use up to 4000 byte frames - so it collects packets (that mean first packet will wait for other to arrive to this frame) and only then sends whole frame out - if you have only ICMP traffic on link, then you will be able to see high ping on some packets (those who was waiting in the frame)

You need other traffic trough the link to see better result.

Re: unstable ping with nstream

Posted: Wed Sep 10, 2008 3:02 am
by ilius168
nice explanation, thank you!

Re: unstable ping with nstream

Posted: Thu Sep 11, 2008 6:53 am
by QpoX
Try and setup a bandwidth test when you ping...

Re: unstable ping with nstream

Posted: Fri Jan 30, 2009 4:17 pm
by ropebih
Now is going 5-6 mbita via that link, but ping is still unstabile
10.10.60.6 64 byte ping: ttl=64 time=2 ms
10.10.60.6 64 byte ping: ttl=64 time<1 ms
10.10.60.6 64 byte ping: ttl=64 time=20 ms
10.10.60.6 64 byte ping: ttl=64 time<1 ms
10.10.60.6 64 byte ping: ttl=64 time=2 ms
10.10.60.6 64 byte ping: ttl=64 time=1 ms
10.10.60.6 64 byte ping: ttl=64 time=1 ms
10.10.60.6 64 byte ping: ttl=64 time=1 ms
10.10.60.6 64 byte ping: ttl=64 time=1 ms
10.10.60.6 64 byte ping: ttl=64 time=1 ms
10.10.60.6 64 byte ping: ttl=64 time=1 ms
10.10.60.6 64 byte ping: ttl=64 time=3 ms
10.10.60.6 64 byte ping: ttl=64 time=1 ms
10.10.60.6 64 byte ping: ttl=64 time=11 ms
10.10.60.6 64 byte ping: ttl=64 time=1 ms
10.10.60.6 64 byte ping: ttl=64 time=20 ms
10.10.60.6 64 byte ping: ttl=64 time=1 ms
10.10.60.6 64 byte ping: ttl=64 time=2 ms
10.10.60.6 64 byte ping: ttl=64 time=1 ms
10.10.60.6 64 byte ping: ttl=64 time=12 ms
10.10.60.6 64 byte ping: ttl=64 time=1 ms
10.10.60.6 64 byte ping: ttl=64 time=1 ms
10.10.60.6 64 byte ping: ttl=64 time=1 ms
10.10.60.6 64 byte ping: ttl=64 time=1 ms
10.10.60.6 64 byte ping: ttl=64 time<1 ms
10.10.60.6 64 byte ping: ttl=64 time=11 ms
10.10.60.6 64 byte ping: ttl=64 time=1 ms
10.10.60.6 64 byte ping: ttl=64 time=1 ms
10.10.60.6 64 byte ping: ttl=64 time=23 ms

Re: unstable ping with nstream

Posted: Fri Jan 30, 2009 6:55 pm
by Mplsguy
ropebih, you should try with latest wireless-test package as that includes improvements discussed here:
http://forum.mikrotik.com/viewtopic.php?f=7&t=27302
this could help latency issue.

Anyway you should be ready for latencies higher than with regular non-nstreme mode and potential jitter. Causes for this can be multiple, but this can be considered the price to pay for increased throughput.

As to the explanation macgaiver gave - it is only partially true: nstreme does not intentionally delay any packets (waiting for more packets to send in one frame). Nstreme starts to aggregate packets when queue of packets waiting to be sent starts to form - that is, when rate at which packets arrive exceeds the capacity of link. This way no latency is added by the sending party. Still aggregating packets actually does increase latency, but this is caused by receive processing - whole big frame must get received (e.g. to check CRC) before further processing as separate packets, and in case aggregated frame is lost, it obviously takes more time to resend.

Re: unstable ping with nstream

Posted: Fri Jan 30, 2009 7:04 pm
by DogHead
Have you tried turning off CSMA? Your current config has it on with polling enabled. We find more consistent delay with CSMA off.

Re: unstable ping with nstream

Posted: Sat Mar 14, 2009 2:28 pm
by ropebih
Hi all,
I figured out when the ping problem starts but I still can't find out what's causing that problem and how can I solve that. I have routerboard 443 and 2x cm9 cards are in that RB (2 links for 2 different towns). Here's the thing: I'm pinging both links, wlan1 and wlan2, and ping is unstabile as I described in my previous posts. After disabling wlan2, ping that goes through link wlan1 is 1ms and it's stabile. That's ideal but as soon as I enable wlan2 ping is broken again. An that's happening only when nstream is on.

If I enable wlan2 and set him on frequency which is not available to client, then client cannot connect and ping to wlan1 is ok, which means that soon as someone is associated on wlan2, ping to wlan1 is broken.

RB433
ROS 3.20
CPU USAGE: 17%
Trafic on link: ~1Mbit

Wlan2 enable:
Image

Wlan2 disable:
Image

Re: unstable ping with nstream

Posted: Sat Mar 14, 2009 5:28 pm
by angboontiong
I also facing this issue and our customer have higher concern on the latency...

Just wonder know, is the mikrotik will have any solution to optimize the ping latency issue?

Re: unstable ping with nstream

Posted: Mon Mar 16, 2009 2:45 am
by jwcn
These ping times are perfectly normal using Nstream.

Re: unstable ping with nstream

Posted: Mon Mar 16, 2009 7:21 am
by JJCinAZ
I agree with jwcn, those ping times with NStream are normal. As I said before in the thread, however, you need to test ping times with a load on the link not just idle. Without NStream, your pretty, non-nstream ping times will go to hell when you load the link. With NStream, a loaded link will be more consistent in latency. If your customer is worried about the ping times and they are paying a LOT of money for it, then use a full-duplex, licensed radio or another solution. You're going to add a varying amount of latency on a half-duplex link.

Re: unstable ping with nstream

Posted: Mon Mar 16, 2009 5:56 pm
by ropebih
I agree with jwcn, those ping times with NStream are normal. As I said before in the thread, however, you need to test ping times with a load on the link not just idle. Without NStream, your pretty, non-nstream ping times will go to hell when you load the link. With NStream, a loaded link will be more consistent in latency. If your customer is worried about the ping times and they are paying a LOT of money for it, then use a full-duplex, licensed radio or another solution. You're going to add a varying amount of latency on a half-duplex link.
Ok, I understand your theory but I think that you are wrong about that. I have large number of nstream links, some of them are even 100 km and they don't have any problems with ping. Read again my last post and tell me why is it ping stabile on wlan1 after I disable wlan2?Both links, wlan1 and wlan2, are using nstream.
I would appreciate if someone from mikrotik can explain what's the problem.

below you can see ping on link with enabled nstream, on 80km
10.10.60.10 64 byte ping: ttl=64 time=2 ms
10.10.60.10 64 byte ping: ttl=64 time=2 ms
10.10.60.10 64 byte ping: ttl=64 time=2 ms
10.10.60.10 64 byte ping: ttl=64 time=1 ms
10.10.60.10 64 byte ping: ttl=64 time=1 ms
10.10.60.10 64 byte ping: ttl=64 time=1 ms
10.10.60.10 64 byte ping: ttl=64 time=2 ms
10.10.60.10 64 byte ping: ttl=64 time=2 ms
10.10.60.10 64 byte ping: ttl=64 time=1 ms
10.10.60.10 64 byte ping: ttl=64 time=2 ms
10.10.60.10 64 byte ping: ttl=64 time=3 ms
10.10.60.10 64 byte ping: ttl=64 time=1 ms
10.10.60.10 64 byte ping: ttl=64 time=1 ms
10.10.60.10 64 byte ping: ttl=64 time=1 ms
10.10.60.10 64 byte ping: ttl=64 time=1 ms
10.10.60.10 64 byte ping: ttl=64 time=1 ms
10.10.60.10 64 byte ping: ttl=64 time=2 ms
10.10.60.10 64 byte ping: ttl=64 time=2 ms
10.10.60.10 64 byte ping: ttl=64 time=1 ms
10.10.60.10 64 byte ping: ttl=64 time=1 ms
10.10.60.10 64 byte ping: ttl=64 time=1 ms
60 packets transmitted, 60 packets received, 0% packet loss
round-trip min/avg/max = 1/1.7/5 ms

Re: unstable ping with nstream

Posted: Tue Mar 17, 2009 1:45 am
by jwcn
How much load is on that link?

Re: unstable ping with nstream

Posted: Tue Mar 17, 2009 2:12 am
by ropebih
about 1 mbit, this is backhaul to RB with two wlan wich are having problems with ping.

Image

Re: unstable ping with nstream

Posted: Tue Mar 17, 2009 5:14 am
by jwcn
What channels are you using?

Re: unstable ping with nstream

Posted: Tue Mar 17, 2009 11:42 am
by ropebih
5280
5520
5825

Re: unstable ping with nstream

Posted: Tue Mar 17, 2009 6:59 pm
by JJCinAZ
tell me why is it ping stabile on wlan1 after I disable wlan2?
Because of self interference? ROS cannot synchronize the transmit on multiple radios, thus you get self-interference. Maybe break the RB433 into two units with some physical separation?

Re: unstable ping with nstream

Posted: Tue Mar 17, 2009 7:54 pm
by nest
I would agree with JJCinAZ. I would investigate the other wlan cards increasing interference. It may be that each time the other wlan card transmits, it interferes with the other. As all the cards are not synchronised with each other, they do not all transmit at the same time. Therefore when one is receiving, one may be transmitting. Therefore by switching off a wlan card, it stops the interference. Try using a new Routerboard and run the link with two separate boxes to prove this.

There is also the possibility that the interference is coming into your antennae or coax from the other system. So you MAY need to separate these as well. However, I think this is unlikely if you are using 30dBm EiRP or less. I only normally saw this problem with carrier grade links and very poor quality coax or poor earthing of ODUs.

Ron.