Hello. Came here looking for advice. I am experimenting with PTMP link with CubeG-5ac60ay-SA as central AP, and 2 clients connected via CubeG-5ac60ay. So far I have only installed first client, which is about 60m away from sector antenna.
Antenna alignment on client’s side seems fairly OK to me: TX Sector 35 TX Sector Info left 0.8 degrees, up 0.8 degrees RX Sector 101 Distance 60.04 m
When running bandwith test between mikrotiks, I do get around ~900Mbit/s on UDP, TCP peaks at around ~300Mbit. But this is as much as it gets. When running rsync between servers, it reaches around 300Mbit tops. But the link gets saturated and the connectivity seems to be breaking when it happens. I also noticed fairly large amounts of Tx Queue Drops - 190 679
Are there any recommendations or best practices to improve stability of the link?
It sounds like they are working as intended. You could try putting some traffic shaping queues, like fq-codel or cake, on the interfaces, limiting the bandwidth to ~900Mbps (or just a bit less) to keep it from saturating the 60GHz interface. It is WiFi-based, after all.
Radio-to-radio TCP tests are going to be limited by the CPU’s of the radios. Try an iperf test between the boxes at both ends.
MikroTik’s 60ghz is pretty damn good in PTP, but PTMP is shite
Have had no luck getting things truly stable, they always disconnect even in relatively short distance links
Speeds are divided amongst number of clients regardless of whether they are actually transmitting or not, and with 8 clients will typically plummet to under 200mbps (UDP will show higher but TCP is slow) making it questionable over simply using 5ghz if its not highly congested
I would guesstimate that this is out of MikroTik’s control and the actual radio chip is handling TCP differently. No amount of tweaking in RouterOS will fix it from my experience. Just need to reduce the number of clients by installing another AP and splitting them up
Good UDP speeds but poor TCP speeds might be an indication of packet loss, possibly due to bursty traffic and too small queues somewhere.
Interestingly, I’ve seen a rise in latency after replacing the AP from the old WAP60G AP running 6.49.x to the new Cube Pro SA running current 7.x, with all stations already being Cube Pro. The location is difficult to access and I was hoping to for an improvement, will probably try to go back to the older device. Not sure if it’s an issue with the newer hardware or just with the newer RouterOS (which can’t be downgraded to 6.x on the new hardware which already comes with 7.x from factory), but I’m almost certain it’s the AP side (stations were replaced one by one as I wanted to move the network to the 66960 channel for longer range, then the AP was replaced last and this is when the bad latencies with high jitter started).
MT also has no single 60GHz device that would be both long-range (like LHG or nRay) and with 5GHz backup (like Cube or Cube Pro), even though the long distances is where the backup is needed most, and it needs to be in the same physical device to use the clever active/passive bonding trick (the backup could even be the 2nd Ethernet port with poe-out for any kind of separate radio, but all devices are 1-port). Don’t even get me started on the promised but never delivered Terragraph… Meanwhile UI Wave released another firmware update supporting up to 31 clients per AP, and nice long range on 69120. But these radios are very damn expensive (only the short-range Wave Pico has reasonable price comparable to MT Cube), it’s yet another vendor lock-in (they only talk to their own radios and forklift replacement of all installed client devices on roofs is not an option) and forces the use of the integrated backup in the AP despite already crowded 5GHz spectrum (MT active/passive bonding trick would be much better, then any already existing AP with 5GHz clients can also be re-used as backup).
All my traffic (TCP UDP or whatever) is wrapped in PPPoE so it shouldn’t matter to the radios. It’s just that TCP is much more sensitive to packet loss and jitter (see the well known formula - https://en.wikipedia.org/wiki/TCP_tuning#Packet_loss - for 10x higher speed you need 100x lower packet loss under the square root, and higher latency with any packet loss also reduces throughput - I have experienced this first-hand several times). I’d suggest to try to replace the Cube Pro SA with the older wAP60G AP (with good old ROS 6.x) and see if that improves things.
It is true. Is there some problem with TCP traffic. UDP is good.
I reported it to support. Next problem is stability with CubeSA - see image.
With WAP60G is speed and MCS good
My MikroTik PTMP network has been pretty good for the past four years, but some of its quirks have cost me customers.
Wave is 100% worth its cost. You can complain all you want, but hundreds of ISPs are deploying it and the money prints itself.
Tachyon is another solution using the same Peraso radios as Wave, but with a bit more flexibility and a backup POE port for your choice of backup radio. You get better range than MikroTik and less cost than Ubiquiti.
