Community discussions

MikroTik App
 
User avatar
macsrwe
Forum Guru
Forum Guru
Topic Author
Posts: 1007
Joined: Mon Apr 02, 2007 5:43 am
Location: Arizona, USA
Contact:

Weird speed problem, bridged network

Thu Jul 22, 2021 7:35 am

Customer's workplace has no Internet provider, so he has installed a small custom all-Mikrotik network to feed Internet from his home, approx. 3km away using a pair of SXT5sq in transparent link mode (b/c), a POE switch in the shop (d), and a small MT router (in bridge mode) to supply WiFi signal (e). The entire network was bridged to his home router (a), which is a Netgear, the only non-MT unit. He ran bridged so he could access all his devices from either location, just like one might to a shed or other outbuilding. All networking units were hardcoded with static IP addresses in 192.168.1.0/24 outside the DHCP range configured in the Netgear. Non-networking units in the shop fetch IP addresses from the home Netgear.

Some time after establishing this link, an envious neighbor in the work complex asked him to share a taste of his Internet; so he set up a second link (f/g) between their two shops, about 1km, still bridged, same addressing rules. Inside the neighbor's shop is a SOHO router, double-NATting, but no one cares. So now the network looks like this:

schematic.png

Traffic works fine, but there is a weird speed issue.

The network will push about 30-45Mbps (which is plenty, as it exceeds his home gateway speed) in each direction over almost any combination of links. The exception is end-to-end: unit b to unit g, which runs spastically around 10Mbps. If speed is measured from c to g, or b to f, full speed is seen. The only time the speed drops is when both wireless links are in play.

I've assured myself that both links are on different channels, so it's not an interference issue. I have turned off STP/RSTP on every device in the network (save possibly the MT POE switch, which has a global RSTP setting that I can't puzzle out, and refuses to upgrade from 2.4). I've assured there are no duplicate IP addresses anywhere in the network. It's not the CPU load of bandwidth test, as I can get fine speed results between any two SXTs other than the two farthest apart. I've disabled g's Ethernet link and run the speed test, just to ensure that nothing going on inside the neighbor's shop is doing something strange, and results are the same. There are only about a half dozen devices in the main shop, so it's not a bridging overload. The wireless distances involved here are quite modest.

Why should the speed drop precipitously only in the end-to-end case? I feel like i'm missing something basic, but I can't find it.
You do not have the required permissions to view the files attached to this post.
 
User avatar
mkx
Forum Guru
Forum Guru
Posts: 11381
Joined: Thu Mar 03, 2016 10:23 pm

Re: Weird speed problem, bridged network

Thu Jul 22, 2021 10:54 am

Can you try UDP throughput test (e.g. using iperf)? I'm guessing that double RTT combined with power save kicking in makes TCP performance drop to floor while UDP performance might remain high. If that's so, you might want to look into WMM priorities...
 
User avatar
macsrwe
Forum Guru
Forum Guru
Topic Author
Posts: 1007
Joined: Mon Apr 02, 2007 5:43 am
Location: Arizona, USA
Contact:

Re: Weird speed problem, bridged network

Thu Jul 22, 2021 11:13 pm

I can't run iperf between MT routers without a non-MT host, which would be significant effort to set up at a site that isn't mine. Plus, I don't understand what I would learn from testing UDP when my problem is TCP speed.

The RTT end to end (b-g) runs a trivial 1ms on nearly every ping, with very occasional single excursions higher as one would expect from wireless.

Being unfamiliar with any "power save" in RouterOS; I did a search and found there was some sort of power save introduced for WMM in 6.30, but WMM is disabled on all wireless links in this network so it shouldn't matter. If there is another, please let me know what it is; I'm not sure how a power-save feature would interfere with a running bandwidth-test once it got running.

I've never done any work with WMM. From what I understand it prioritizes video/voice/data, but you have to characterize the packets yourself to begin with. That's always been more detail than any customer of mine has required. And I don't understand how it would help here, since video and voice are neither the speed problem nor a priority with the customer.

EDIT: I will add, running the UDP option in bandwidth test gives similar results to the TCP option; the numbers are slightly higher (~15 end-to-end compared to ~45 otherwise) but the difference is still there.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11967
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Weird speed problem, bridged network

Thu Jul 22, 2021 11:28 pm

I read all posts.

What RouterOS version is used?
All the same?

The bridge are on WDS on on bridge mode or others? (see point 1)

I can think two tings:

1) better see the exports of all 5 devices

2) when tested with only one wifi at time, no problem,
when tested involving both wifi at same time, both radio disrupt others radio signal for ROS and interferencies.
Probably physically too close and frequencies not with one hole of at least 60MHz between or overlapped like 5500 Ceee / 5600 eeeC etc.
 
User avatar
macsrwe
Forum Guru
Forum Guru
Topic Author
Posts: 1007
Joined: Mon Apr 02, 2007 5:43 am
Location: Arizona, USA
Contact:

Re: Weird speed problem, bridged network

Thu Jul 22, 2021 11:57 pm

ROS is all 6.45.1
Bridges are in transparent mode (bridge/station-bridge).
b-c using 5230/20/an, f-g using 5220/20/an.
I thought I was guaranteed no mutual interference between single 5GHz channels.
I found a hole at 5200 and threw f-g down there, and the problem went away.
I learned something new today. Thanks for your help!
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11967
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Weird speed problem, bridged network

Fri Jul 23, 2021 12:01 am

Well done, thanks.

But really all the credit goes to you,
without your precise description of the problem, I would never have had that intuition.
 
User avatar
mkx
Forum Guru
Forum Guru
Posts: 11381
Joined: Thu Mar 03, 2016 10:23 pm

Re: Weird speed problem, bridged network

Fri Jul 23, 2021 8:28 am

b-c using 5230/20/an, f-g using 5220/20/an.
I thought I was guaranteed no mutual interference between single 5GHz channels.

ROS lets one set things which are not exactly according to standards / best practice. If you check the list of 5GHz channels you'll see that valid channel frequencies for 20MHz channels are 5220MHz (channel 44) and 5240MHz (channel 48) while frequency 5230MHz (channel 46) is valid for 40MHz channel. However, with Mikrotik notation of channels - 40 MHz channels are actually 20+20MHz channels with channel setup Ce or eC - one should still use the 20MHz channel frequencies when setting channels for any channel width.

BTW, your UDP tests (being almost as bad as TCP) proved that my theory about delays was wrong. For that part the test still proved useful even though most traffic over link is TCP.
 
User avatar
macsrwe
Forum Guru
Forum Guru
Topic Author
Posts: 1007
Joined: Mon Apr 02, 2007 5:43 am
Location: Arizona, USA
Contact:

Re: Weird speed problem, bridged network

Fri Jul 23, 2021 9:56 am

Holy ****! That went right over my head! The customer did that, and I never even noticed. (My recommendation of 5320 got transposed at one point to 5230 and that was the number he ran with. I would have noticed the lack of bold type; he didn't know what it meant.) So the frequencies were actually overlapping after all! I was under the impression that RouterOS would simply refuse to use a half-channel frequency, so I never even checked for it.

I just reached out and reset his home link to 5240, which works just as well for him if not better, and doesn't cross channels.

Thanks for catching this.

Who is online

Users browsing this forum: Amazon [Bot], cyrq, gigabyte091, maigonis, normis and 32 guests