i am on a DOCSIS 3.1 Network from VODAFONE-GERMANY. I have 500 MBit/s via UDP Speedtest (BW-Test) and get only 50-80 MBit/s in TCP with one single concurrent socket.
Also Downloading a ISO-File with /tools/fetch is limited at 80 MBit/s.
First i wonderd if the DOCSIS-Channel is full of data, but UDP-Tests get 500 MBit/s on hEX S. So the DOICSIS over-subscription can not be the issue.
I think that the packets are out of order so the CPU have to rebuild the data, but hEX S have not much memory and cpu to get the correct information on time, so the TCP-Stack is limiting the bandwith.
But how can i confirm my thougths? Is there any tool on mikrotik to proofe that i have an TCP reordering / out of order issue?
What i did test:
Because UDP performs so good, i established a UDP-Tunnel via VxLAN ENCAP via UDP.
Doing Speedtest via UDP over the tunnel, it gets full speed of 500M (100% CPU Usage on hEX S).
Doing the same Test via TCP over the tunnel, i get limited at 80M (40% CPU Usage on hEX S).
Reordering of TCP packets (if they are indeed delivered out-of-order) is done by final receiver (so typically it’s not router’s job to do it). TCP out-of-order delivery can affect throughput (if TCP stack NACKs packets which actually arrive a bit later … which makes sender’s transmit window shrink). But there are things which can affect TCP throughput (and are more likely to happen), such as long RTT or large RTT jitter (both require larger Tx window but sender my not be able to increase it to required size). Or router’s CPU overload, ROS will process packets, belonging to single connection, with same CPU core (on multi-core devices this may severely limit single-connection throughputs) … run tool->cpu profiler and verify if one of CPU cores is maxed out (100% load may shift from one core to another with time)
The best tool to look into any issues is wireshark, run on your test PC … it’ll show you if packets are delivered out-of-order, if there are any missing packets, etc.