Very poor bandwith between two SXT2

shunkica · Fri Nov 19, 2021 7:26 pm

I have two SXT2 with LOS at around 190m and the throughput is horrible at around 10Mbps.
Both are running v6.48.5.

/tool bandwidth-test address=192.168.99.1 direction=transmit  duration=30 protocol=udp
                status: done testing
              duration: 31s
            tx-current: 13.5Mbps
  tx-10-second-average: 13.3Mbps
      tx-total-average: 9.3Mbps
           random-data: no
             direction: transmit
               tx-size: 1500
      connection-count: 20
        local-cpu-load: 8%
       remote-cpu-load: 11%

The station signal strength is around -69/-58.
Setup is N-only and 20/40 XX, with auto frequency.

/interface wireless
set [ find default-name=wlan1 ] band=2ghz-onlyn channel-width=20/40mhz-XX country=croatia disabled=no frequency=auto hide-ssid=yes mode=ap-bridge ssid=link123 wps-mode=disabled
/interface wireless security-profiles
set [ find default=yes ] authentication-types=wpa2-psk eap-methods="" mode=dynamic-keys supplicant-identity=MikroTik wpa2-pre-shared-key=**********

Is there anything I can do to improve the throughput here other than replace the devices?

bpwl · Fri Nov 19, 2021 9:47 pm

Please check if you can use a 40 MHz wide channel. This is not very often the case, as the 2.4GHz is normally crowded.

MT devices are not smart-auto ! Set it yourselves (and no XX as this is undefined) after check with scan/snooper/freq-usage for the clearest channel.

Amm0 · Fri Nov 19, 2021 11:01 pm

While the CPU looks low, Mikrotik never recommends running the bandwidth test on the SUT. Not sure that issue here.

As suggested, try using 20Mhz and setting the channel explicitly. Might also want to try a lower MTU, say 1400, in the Bandwidth Test, and see what that does to your speed – although pretty it sure mtu=1500 between over link (but perhaps not).

shunkica · Fri Nov 19, 2021 11:32 pm

I set the channel width to 20Mhz and selected the frequency that gives me the highest throughput on test,and got an increase of about 20 Mbps, but this is still not even close to what I would like to achieve ( 80 Mbps )

/tool bandwidth-test address=192.168.99.1 protocol=udp direction=transmit duration=15s
                status: done testing
              duration: 15s
            tx-current: 29.0Mbps
  tx-10-second-average: 30.4Mbps
      tx-total-average: 28.8Mbps
           random-data: no
             direction: transmit
               tx-size: 1500
      connection-count: 20
        local-cpu-load: 24%
       remote-cpu-load: 14%

Here are the wifi stats:

/interface wireless registration-table print stats
 0 interface=wlan1 radio-name="*****" mac-address=***** ap=yes wds=no bridge=yes rx-rate="104Mbps-20MHz/2S" 
   tx-rate="78Mbps-20MHz/2S" packets=39020,3142 bytes=55737298,3352178 frames=39020,3177 frame-bytes=55816276,3340390 hw-frames=53631,3440 
   hw-frame-bytes=77212888,3610651 tx-frames-timed-out=0 uptime=2m38s last-activity=70ms signal-strength=-58dBm@6Mbps signal-to-noise=55dB 
   signal-strength-ch0=-63dBm signal-strength-ch1=-60dBm tx-signal-strength-ch0=-79dBm tx-signal-strength-ch1=-65dBm 
   strength-at-rates=-66dBm@1Mbps 2m35s990ms,-65dBm@2Mbps 2m34s660ms,-66dBm@5.5Mbps 2m33s20ms,-62dBm@11Mbps 2m29s910ms,-58dBm@6Mbps 80ms,-
                  64dBm@9Mbps 2m33s140ms,-64dBm@12Mbps 2m29s910ms,-64dBm@18Mbps 2m27s660ms,-64dBm@24Mbps 1m45s840ms,-63dBm@36Mbps 
                  1m48s200ms,-63dBm@48Mbps 1m43s40ms,-63dBm@HT20-0 2m35s680ms,-64dBm@HT20-1 2m1s390ms,-64dBm@HT20-2 2m2s410ms,-63dBm@HT20-3 
                  1m21s310ms,-59dBm@HT20-4 8s100ms,-58dBm@HT20-5 80ms,-57dBm@HT20-6 4s90ms,-58dBm@HT20-7 9s10ms 
   tx-signal-strength=-65dBm tx-ccq=85% rx-ccq=85% p-throughput=53459 distance=8 nstreme=no framing-mode=none routeros-version="6.48.5" 
   last-ip=192.168.99.1 802.1x-port-enabled=yes authentication-type=wpa2-psk encryption=aes-ccm group-encryption=aes-ccm 
   management-protection=no compression=no wmm-enabled=yes tx-rate-set="CCK:1-11 OFDM:6-54 BW:1x HT:0-15"

It is not just the bandwidth test that is the problem. Actual bandwidth through the link is even worse than the values from bandwidth test.

I also tried changing tx-size to 1400 but that made no difference.

bpwl · Sat Nov 20, 2021 12:09 am

/interface wireless registration-table print stats
rx-rate="104Mbps-20MHz/2S"
tx-rate="78Mbps-20MHz/2S"
frames=39020,3177 hw-frames=53631,3440
signal-strength=-58dBm@6Mbps signal-to-noise=55dB
signal-strength-ch0=-63dBm signal-strength-ch1=-60dBm tx-signal-strength-ch0=-79dBm tx-signal-strength-ch1=-65dBm
strength-at-rates=-58dBm@HT20-7 9s10ms
tx-signal-strength=-65dBm
tx-ccq=85% rx-ccq=85% p-throughput=53459
authentication-type=wpa2-psk encryption=aes-ccm group-encryption=aes-ccm
tx-rate-set="CCK:1-11 OFDM:6-54 BW:1x HT:0-15"

Lets read this .... most is good to very good

rx-rate="104Mbps-20MHz/2S"
tx-rate="78Mbps-20MHz/2S"

Ok that's the interface rate, data rate through this is normally limited to 50-60% because of unavoidable 802.11 overhead. So 50-40 Mbps is expected max.
Dual stream (2S) is used. Theorethical max is 144Mbps interface rate (but that needs SGI, else here it is 130Mbps). 80 Mbps data rate is almost impossible.

frames=39020,3177 hw-frames=53631,3440

We have re-transmits. Some data packages are sent multiple times before they are acknowledged.
This reduces the effective data throughput here to 39/53 of the max possible above 39/53 * 50 = rx 36 Mbps - tx 29 Mbps
This is also the reason why the interface rate is reduced from MCS15 down till some more stable encoding is reached. (MCS 13 and 12 : see http://mcsindex.com/ )

Signal strength is OK

tx-ccq=85% rx-ccq=85% p-throughput=53459

CCQ Calculated from other statistics. p-throughput looks optimistic (did it not account for retransmits?) 53Mbps expected max as per RouterOS. Will be reduced by retransmits.

Rest is OK.

Not available with RouterOS is info about the contention for air-time/transmit time. (% channel busy) This will also again reduce the effective max data rate. (Co-channel coexistence/interference)
(Freq Usage: could be an indication for channel busy, but this is not real time). High signal to noise suggests here no adjacent channel interference.

Real usage (not only UDP test)

Bi-directional will split the airtime and data rate for tx and rx. TCP is always somewhat bi-directional (ACK must be sent)
TCP congestion avoidance (in the client device) might further reduce the TCP performance.

Amm0 · Sat Nov 20, 2021 12:21 am

Is there clear 40Mhz slot available, could try that again, but set the channel based on the freq. scan data?

Also, not sure at 2.4Ghz, but Nv2 protocol is sometimes used for PtP links.

shunkica · Mon Nov 22, 2021 11:10 pm

I managed to get 170 Mbps throughput by setting manual-txpower and using 2427 Ce

Freq monitor:

 /interface wireless frequency-monitor duration=60s
number: 0
         FREQ          USE         NF
      2412MHz          10%        -97
      2417MHz         8.3%       -112
      2422MHz         5.9%       -113
      2427MHz         4.3%       -113
      2432MHz         5.3%       -113
      2437MHz         7.7%       -113
      2442MHz           9%       -113
      2447MHz        11.6%       -113
      2452MHz         6.2%       -112
      2457MHz        10.1%       -112
      2462MHz        19.9%       -112

Spectral history:

bpwl · Tue Nov 23, 2021 12:13 am

Crowded environment. The 'Next' step then is using "NV2".
(Has already been given as suggestion by @Amm0)

Some more information here: viewtopic.php?t=179942#p890126
(But it is the "advanced" step, so first the channel information was needed to get to this trial.)

shunkica · Tue Nov 23, 2021 12:24 am

It's actually a PtmP setup and one of the clients is a unifi antenna so that's why I'm not using Mtik proprietary protocols.
I will try NV2 and if that gives me good results I will consider replacing the unifi antenna with one of the SXT2 and replace the SXT2 AP with a BaseBox2.

EDIT: I tried NV2 and the throughput was even worse than 802.11n (110-125 Mbps on 802.11n vs 80-100 Mbps on NV2)

bpwl · Tue Nov 23, 2021 12:37 am

I don't know your setup. But with the Basebox you still have to select the best antenna.
The gain of both sides adds up for the signal strength! Actually the transmit power is legally limited, but the gain is also used to ampltfy the received signal.
If the PtMP is not a 360° coverage, you might be better off with some directional unit. (MantBox ?).
It will also eliminate some interference from the non-covered directions.

bpwl · Tue Nov 23, 2021 12:40 am

EDIT: I tried NV2 and the throughput was even worse than 802.11n (125Mbps on 802.11n vs 81.5Mbps on NV2)

Same channel and width ??? Expected 160Mbps.

shunkica · Tue Nov 23, 2021 12:46 am

The angle between the two closest stations is 50 degrees when looking from the AP, but they are at 30 and 50 meters from the AP, so it is not an issue.
The one I am measuring from is 190 meters away, but the AP is pointing directly at that one, since it is between the other two stations.

shunkica · Tue Nov 23, 2021 12:48 am

EDIT: I tried NV2 and the throughput was even worse than 802.11n (125Mbps on 802.11n vs 81.5Mbps on NV2)
Same channel and width ??? Expected 160Mbps.

I left everything exactly the same between the two measurements. The only thing I changed was the wireless protocol from 802.11 to NV2.
I even went back and forth twice to confirm, and 802.11 was faster twice, with bandwidth test settings direction=receive protocol=udp duration=60s

shunkica · Tue Nov 23, 2021 1:06 am

Channel: 2427/20-Ce/gn(10dBm)

direction=both protocol=tcp
802.11
tx-total-average: 24.6Mbps
rx-total-average: 25.2Mbps
nv2
tx-total-average: 14.3Mbps
rx-total-average: 14.8Mbps

direction=receive protocol=tcp
802.11
rx-total-average: 79.0Mbps
nv2
rx-total-average: 57.3Mbps

direction=both protocol=udp
802.11
tx-total-average: 26.3Mbps
rx-total-average: 42.3Mbps
nv2
tx-total-average: 12.5Mbps
rx-total-average: 53.3Mbps

direction=receive protocol=udp
802.11
rx-total-average: 143.5Mbps
nv2
rx-total-average: 115.2Mbps

bpwl · Tue Nov 23, 2021 2:25 am

Sorry, can't help you further. Results do not make sense to me, I cannot explain them.
I assume you use the default NV2 parameters (2ms TDMA, dynamic downlink, Downlink ratio 50 ....)

But:
-I never used SXT 2 on 2.4GHz (but many SXTsq 5 ac)
-RouterOS 6.48.5 is very new, and as such unknown to me for performance
-all channels from 4 till 11 are used, and they are busy with strong signal (red). There may be quite some retransmits.

EDIT: when set on channel 4Ce , channels used are from 2 till 10, actually interfering with all 3 common 1-6-11 settings.

Expected :
- UDP: both directions to be 45% of one direction (Here it drops to less than 30% for 802.11)
- more symmetric results tx/rx as both sides are SXT
- for nv2 , 60% of theoretical max interface rate. Max Interface rate is 300Mbps for 40MHz/2S/SGI. (I get 260Mbps out of 400Mbps interface rate with SXTsq 5ac)
- indeed lower speed for TCP (congestion avoidance)

Amm0 · Tue Nov 23, 2021 6:07 am

Agree with @bpwl... Basically the 2.4Mhz is tough. Pretty sure you're aware that you could re-architect this to avoid 2.4Ghz, but get trying to "make it work"... but you may be near the end of the line using that band where you're at...

UDP being faster than TCP is totally expected. Now more radical differences between the UDP and TCP speeds, however would be indicative of link errors/retrinsmits since TCP will slow down. (See @bpwl comment about TCP congestion control)

Perhaps the NV2 may not be best in a congested 2.4Ghz – and seemed like you needed 802.11 for PtMP. Could go back to 802.11 mode, maybe try 2437 (Channel 6)? Since the alternative to NV2, is "playing nice" in 802.11, that would include using 1,6,11. Think is that could reduce errors on your neighbors – basically since errors result in just more RF congestion (e.g. same data has to be transmitted again after an error), thus wasting more time/spectrum - so slamming others traffic doesn't help as much as one might think. i.e. "congestive collapse".

I guess you could try 7.1rc6 to see if anything there helps. Unlikely, but not hard to try. Similarly going back to 6.47.10 (previously "long-term") may be worth a shot. But the your RF situation seem most likely the actual issue – now is there some magic setting to help more? maybe – but you'd likely want to look at the RF using a real (or SDR) spectrum analyzer to get a better look at the RF than via the AP's radio.

Using a queue on the link may help, since your speeds do seem more off than the RF indicates. v7.1 has fq_codel and/or cake which may be useful. But not sure that be some panacea here given 2.4Ghz is just more likely to have actual congestion/errors/retransmits.

bpwl · Tue Nov 23, 2021 11:44 am

Yep as @Amm0 says, we are in a different ball-game here. Massive interference and fluctuating throughput. The strategies are different, and only many trial and error will show what helps and what is contra-productive. The CCQ is a good indication for any progress or regression.

Just some thoughts:
- SXT is amplifying all contending transmitters in it's receive channel. The 60° beam amplification is actually only reduced to 7dBi at the edge, so all what is within 120° can be very disturbing for the receiver. AP and client are equally important in this.
- poor signal quality will make MCS rate fluctuate. TCP congestion avoidance could overreact. Capping the max MCS might bring stability. (http://mcsindex.com) . Like removing all higher qam MCSes. Like removing MCS 5-6-7 and MCS 13-14-15. It will limit the max interface speed to 180Mbps, good for 90Mbps UDP and some lower TCP. But MCS 5 and 13 may work as well.
- reducing A-MSDU, for smaller packets with checksum. (2300bytes and much lower). A-MPDU will assemble those small checksummed A-MSDU, and retransmit only the broken ones.
- enabling "Adaptive noise immunity" for reducing the receive sensitivity (but the freq scan showed strong transmitters)
- are you in a country that allows channels 12,13,14 ?

But I would try to escape this jungle.
-SXTsq 5 ac works in the 5GHz band. Spectrum could be less crowded, but this must be checked. The angle is also 12°. For PtMP you will need the SXT SA5 as AP with license 4 and 60° beam.
-60GHz is een a further escape. (I get 2310Mbps between Cube, but aligning is quite a challenge, and goes better with a laser pointer.

shunkica · Tue Nov 23, 2021 5:19 pm

Agree with @bpwl... Basically the 2.4Mhz is tough. Pretty sure you're aware that you could re-architect this to avoid 2.4Ghz, but get trying to "make it work"... but you may be near the end of the line using that band where you're at...

My reasoning is that everyone is slowly switching to 5Ghz so in time I will have the 2.4Ghz band all to myself

I guess you could try 7.1rc6 to see if anything there helps.

I have considered this but if something goes wrong I will have to go up on the roof and this is not something I want to do during the winter months

But I would try to escape this jungle.
-SXTsq 5 ac works in the 5GHz band. Spectrum could be less crowded, but this must be checked. The angle is also 12°. For PtMP you will need the SXT SA5 as AP with license 4 and 60° beam.

I really don't want to change my hardware. It is "good enough" for what I use it for.

I appreciate the input but I have found that if I keep the channel in the dip between 1 and 6 (2427) I can just about max out my uplink bandwidth, so I will keep it there for now unless something changes, especially because most of the stuff about MCS and MSDU goes straight over my head.

Amm0 · Tue Nov 23, 2021 6:36 pm

That wait for now newer IoT things that still use 2.4Ghz to die may be a while... so I would listen @bpwl re the "MCS stuff". That advice is pretty easy: you set Wireless>(interface)>Data Rates to "configured" and then uncheck the boxes to remove the higher speed ones as suggested above (MCS 5-6-7 and MCS 13-14-15). This tells the radio "don't even allow this higher speed, even if the other side requests it" when you uncheck the "higher order" MCS values. This will make the link more "useable" I think, even if you're give up what appears to be speed (since higher speeds are transitory, it not all that helpful).

Also, since you reported..

Actual bandwidth through the link is even worse than the values from bandwidth test.

This is why @bpwl esstenially suggests "capping" the speeds (by unchecking ). You really want a stable speeds, even if the link can sometimes do more. Basically "sometimes" isn't all that helpful, and likely not usable by clients, since it's generally at the expense of more errors/retransmits. It the later that really cause problems with TCP. This effect of errors/retransmits gets worse downstream since presume some of the clients on either of the link may also be on Wi-Fi, which too may have a small amount of errors/jitter. At the end of the day, changes in latency (which is what happens with successful retransmits in crowded spectrum) will keep TCP so far away from the line speed, certainly errors will, so trying to "squeeze" more out of the RF can be counter-productive, if it comes at the expense of quality...
...thus @bpwl focus "CCQ" as a good, quick, single metric on how a channel/config is going.

For similar reasons, this is why I suggest using a shaping queue on either of the link to shape the traffic to a speed the link is actually stable at will help your client's speeds – you've been running tests on the APs themselves, but once more than a few people try using the link, a simple queue may help get speeds closer to the router on the devices.
...Basically the variable speed on the link is why the speeds are even worse downstream of your 2.4Ghz link.

I guess you could try 7.1rc6 to see if anything there helps.
I have considered this but if something goes wrong I will have to go up on the roof and this is not something I want to do during the winter months

Yes I'd wait for spring to try v7.1 & really not sure any there would help the 2.4Ghz link side. But your queuing/shaping options do improve a lot, which is why I half suggested it. But those are still pretty new and under-development – you likely don't ROS bugs in mix here.

Amm0 · Tue Nov 23, 2021 6:45 pm

One tip here, since I see your using "Bandwidth Test", but there is /tool speed-test in the ROS Terminal. This shows the jitter on the link and does both TCP and UDP test at same time. There are some options for duration. It's pretty useful to tweak-and-test these things, since it runs the same test/length everytime so you get more apples-to-apples results. I have no idea why it's not just in winbox.

shunkica · Tue Nov 23, 2021 11:47 pm

That wait for now newer IoT things that still use 2.4Ghz to die may be a while... so I would listen @bpwl re the "MCS stuff". That advice is pretty easy: you set Wireless>(interface)>Data Rates to "configured" and then uncheck the boxes to remove the higher speed ones as suggested above (MCS 5-6-7 and MCS 13-14-15). This tells the radio "don't even allow this higher speed, even if the other side requests it" when you uncheck the "higher order" MCS values. This will make the link more "useable" I think, even if you're give up what appears to be speed (since higher speeds are transitory, it not all that helpful).

I disabled MCS 5-6-7 and 13-14-15 as suggested and even though my tests with /tool speed-test are inconclusive (because of all the interference and inconsistent results) I estimate there is a slight increase in TCP download speed and some reduction in avg ping and jitter.

...thus @bpwl focus "CCQ" as a good, quick, single metric on how a channel/config is going.

Tx/Rx CCQ goes from around 30% to 90%, but I'd estimate most of the time it stays around 60%.

Very poor bandwith between two SXT2

Very poor bandwith between two SXT2

Re: Very poor bandwith between two SXT2

Re: Very poor bandwith between two SXT2

Re: Very poor bandwith between two SXT2

Re: Very poor bandwith between two SXT2

Re: Very poor bandwith between two SXT2

Re: Very poor bandwith between two SXT2

Re: Very poor bandwith between two SXT2

Re: Very poor bandwith between two SXT2

Re: Very poor bandwith between two SXT2

Re: Very poor bandwith between two SXT2

Re: Very poor bandwith between two SXT2

Re: Very poor bandwith between two SXT2

Re: Very poor bandwith between two SXT2

Re: Very poor bandwith between two SXT2

Re: Very poor bandwith between two SXT2

Re: Very poor bandwith between two SXT2

Re: Very poor bandwith between two SXT2

Re: Very poor bandwith between two SXT2

Re: Very poor bandwith between two SXT2

Re: Very poor bandwith between two SXT2

Who is online