Community discussions

MikroTik App
 
Crabbotron
just joined
Topic Author
Posts: 13
Joined: Fri Jan 20, 2023 7:06 am

RB5009UPr+S+ Bandwidth Issue

Fri Jan 27, 2023 5:50 pm

Hello MikroTik friends!
I come to you in an hour of great need. I have seen similar posts (and hope that I didn't happen to skim over a matching one), but not one that seems to scratch my itch.

The issue here seems to be that I cannot get my full ISP-provided bandwidth through my shiny new RB5009. I am getting about 75-100 Mbps download, 5-6 Mbps upload, when I have a dedicated 1G line to the house, and have previously seen 940/940 (gotta account for overhead). Below is a list of testing and information.

Troubleshooting accomplished:
- Upgraded to 7.7
- Tested with laptop directly from fiber ONT, got 940/940 (ish, but usually right around there)
- Tried moving from 2.5G (ether1) to 1G (ether8), this is where the config currently is
- While on ether1, I tried removing the 2500 negotiation option (based on other forum posts I've seen, did not fix it)
- Moved fiber ONT to static 1000FDX, as well as RB5009 (I work in the ISP NOC, so I've tried it all)

Information:
- The cables between the fiber ONT and RB5009 are CAT6a, and are certified to 10G (but they are literally 6 inches long).

I'm sure I missed some necessary information (and I sure hope y'all are nicer than StackOverflow). I have also attached my `/export hide-sensitive`, with my static IP information redacted, but let's all pretend that it's working (I can assure you that if it was a static issue, the ISP router would just not let it connect at all, instead of torturing me with slow speeds).

Editing to add:
If there are any lovely MikroTik support staff here, could you please look at my ticket? SUP-105349 Thanks in advance!
You do not have the required permissions to view the files attached to this post.
Last edited by Crabbotron on Sat Jan 28, 2023 8:40 am, edited 1 time in total.
 
kowal
newbie
Posts: 30
Joined: Sun Jul 06, 2014 2:23 am

Re: RB5009UPr+S+ Bandwidth Issue

Sat Jan 28, 2023 2:52 am

Do you have any drops/error on WAN interface?
I've also figured that you have disabled hw-offloading on some ports, is this for some reason?
/interface bridge port
add bridge=bridge comment=defconf hw=no interface=ether2
add bridge=bridge comment=defconf hw=no interface=ether3
add bridge=bridge comment=defconf hw=no interface=ether4
add bridge=bridge comment=defconf hw=no interface=ether5
add bridge=bridge comment=defconf hw=no interface=ether6
add bridge=bridge comment=defconf hw=no interface=ether7
P.S. remove your serial number from that export file
 
Crabbotron
just joined
Topic Author
Posts: 13
Joined: Fri Jan 20, 2023 7:06 am

Re: RB5009UPr+S+ Bandwidth Issue

Sat Jan 28, 2023 8:40 am

Do you have any drops/error on WAN interface?
I've also figured that you have disabled hw-offloading on some ports, is this for some reason?
/interface bridge port
add bridge=bridge comment=defconf hw=no interface=ether2
add bridge=bridge comment=defconf hw=no interface=ether3
add bridge=bridge comment=defconf hw=no interface=ether4
add bridge=bridge comment=defconf hw=no interface=ether5
add bridge=bridge comment=defconf hw=no interface=ether6
add bridge=bridge comment=defconf hw=no interface=ether7
P.S. remove your serial number from that export file
I do not have any errors on the WAN interface.
As for the HW-offload on those bridge ports, when I was setting up the VLAN bridge on sfp-sfpplus1, it didn't have any HW-offload, but once I disabled HW-offload on the other ports, the VLAN bridge was able to have it. I am doing router-on-a-stick, so I have no need for the 1G ports, I just need a WAN port, and the VLAN trunk to my core switch.
Is there something awful that can be done with a serial number? I was only aware that it served to identify my specific board. Even still, I will edit my original post to remove it.
 
User avatar
mkx
Forum Guru
Forum Guru
Posts: 11381
Joined: Thu Mar 03, 2016 10:23 pm

Re: RB5009UPr+S+ Bandwidth Issue

Sat Jan 28, 2023 10:43 am

Re serial number: if /ip/clud property ddns-enabled is set to yes (seems like default is no, but anyway), then every device will have DNS entry in format of <serial number>.sn.mynetname.net ... and if somebody notices severely botched firewall setup, it's only too easy to launch a direct attack against such device. MAC addresses are, OTOH, not such a large attack surface, one has to be in same L2 segment to be able to use that knowledge.

Re bridges: since you're using router in a RoaS manner, you don't really need bridge trunk-bridge, you could anchor your vlanXX interfaces directly on sfp-sfpplus1 interface. There's no bridge functionality that you're using in trunk-bridge ... and that includes VLAN functions. And very probably L3HW offload (I don't think RB5009 can offload routing to built in switch chip).

When trying to shuffle data, try to run CPU profiler and see if any of CPU cores is highly loaded ... and if it is, which process is loading it?

A try you can do quite easily: replace your 6 inch patch cables with something decently longer and of lower category, cat5e should be fine on short stretch ... not very common, but it could be that one of partner ports (either RB5009 or the other end) uses too high Tx power on that RJ45 port and the other end's Rx path has problems receiving that. As I wrote, not very likely (I'd expect that if this was happening, there would be Rx errors conted), but trying this theory is trivial.

BTW, I'm not sure how you're using RB5009 as RoaS ... you're saying that only used port connects to ONT. So how are those private VLANs then forwarded to your LAN? Is ONT used as a switch for your LAN or what?
 
Crabbotron
just joined
Topic Author
Posts: 13
Joined: Fri Jan 20, 2023 7:06 am

Re: RB5009UPr+S+ Bandwidth Issue

Sat Jan 28, 2023 11:24 am

New config file attached, bridge has been disabled (for now, will delete later). Still seem to be getting the same results.

When trying to shuffle data, try to run CPU profiler and see if any of CPU cores is highly loaded ... and if it is, which process is loading it?
I ran the profiler while moving some files around on the LAN, downloading files on multiple computers/servers, watching a Discord stream, and running a speedtest. The highest utilization I saw was ~4.5% on cpu3, and the total CPU usage never went above 7%.

BTW, I'm not sure how you're using RB5009 as RoaS ... you're saying that only used port connects to ONT. So how are those private VLANs then forwarded to your LAN? Is ONT used as a switch for your LAN or what?
I should have clarified, and maybe RoaS is not the proper term to my setup, I've never been good with the lingo. ether8 goes to the ONT, which is basically serving as a media convertor as well as VLAN tagger (but that is all ISP-level stuff, not anything that affects my setup. I just pass the untagged traffic to the ONT, and the rest is PFM internet things). sfp-sfpplus1 goes to my core switch, which is a CRS326. From there, the L2VLANs are broken down based on where they need to go.

Some other things that may or may not be important:
- You'll see that the L2MTU on ether8 has been changed to 2000, this is to match the ONT
- Forgot to mention this, but there is also a dumb switch between the ONT and RB5009. However, I get the same results whether the RB5009 is plugged into the ONT or the switch. Laptop testing done on both the switch and ONT get the same results as well, which is the full speed.
- The other router on the dumb switch (see my amazingly perfect network diagram) gets the full speed with the current config, so it's not an ONT or switch issue

Edit to add:
The computer I am testing from has a 10G link to the CRS326. iperf testing within the LAN to servers that are also connected to the CRS326 (albeit on different VLANs) shows a full 10G within the LAN, so the issue here seems to purely just be WAN <-> LAN, LAN <-> LAN is unaffected.
You do not have the required permissions to view the files attached to this post.
 
User avatar
jvanhambelgium
Forum Veteran
Forum Veteran
Posts: 985
Joined: Thu Jul 14, 2016 9:29 pm
Location: Belgium

Re: RB5009UPr+S+ Bandwidth Issue

Sat Jan 28, 2023 11:39 am

Reset the L2MTU value on the RB5009 again to default value and try again ?
What effect does this have ?
 
Crabbotron
just joined
Topic Author
Posts: 13
Joined: Fri Jan 20, 2023 7:06 am

Re: RB5009UPr+S+ Bandwidth Issue

Sat Jan 28, 2023 11:49 am

Reset the L2MTU value on the RB5009 again to default value and try again ?
What effect does this have ?
Reset to 1514, and there appears to be no change.
 
biomesh
Long time Member
Long time Member
Posts: 561
Joined: Fri Feb 10, 2012 8:25 pm

Re: RB5009UPr+S+ Bandwidth Issue

Sat Jan 28, 2023 3:33 pm

Do you get full speed if you test from a device with only a 1 gig nic?
 
Crabbotron
just joined
Topic Author
Posts: 13
Joined: Fri Jan 20, 2023 7:06 am

Re: RB5009UPr+S+ Bandwidth Issue

Sat Jan 28, 2023 5:46 pm

Do you get full speed if you test from a device with only a 1 gig nic?
Just tested, and sadly no. I plugged a laptop directly into ether2, and got the same results as before.
 
User avatar
jvanhambelgium
Forum Veteran
Forum Veteran
Posts: 985
Joined: Thu Jul 14, 2016 9:29 pm
Location: Belgium

Re: RB5009UPr+S+ Bandwidth Issue

Sat Jan 28, 2023 8:34 pm

So for my understanding, the "WAN" interface is configured just to obtain via DHCP a IP-address from the ISP, no PPPoE anymore right ?
Really, really weird phenomena you have with RB5009.
Did you reboot after setting the MTU back to default value ?

Could you perform a complete factory-reset and don't touch the L2MTU and perform a minimal config just to get things working?

Since you also have tested without the "dumb" switch in between, that can be ruled out too I guess.
 
Crabbotron
just joined
Topic Author
Posts: 13
Joined: Fri Jan 20, 2023 7:06 am

Re: RB5009UPr+S+ Bandwidth Issue

Sun Jan 29, 2023 6:14 am

So for my understanding, the "WAN" interface is configured just to obtain via DHCP a IP-address from the ISP, no PPPoE anymore right ?
I have a static IP address assigned by the ISP, which is manually programmed into the interface (we don't use DHCP reservations for statics).

Did you reboot after setting the MTU back to default value ?
You know, in all of my troubleshooting, I have forgotten to do reboots, outside of doing factory resets. However, I did do the factory reset your prescribed, which involved rebooting, and still no change.

Could you perform a complete factory-reset and don't touch the L2MTU and perform a minimal config just to get things working?
I have done this, and there is no change. After resetting, and simply just throwing in the static IP config, I am getting the same results.
 
holvoetn
Forum Guru
Forum Guru
Posts: 5321
Joined: Tue Apr 13, 2021 2:14 am
Location: Belgium

Re: RB5009UPr+S+ Bandwidth Issue

Sun Jan 29, 2023 8:56 am

To rule out the modem
Can you connect pc or so directly to modem, setting the ip manually ?
 
Crabbotron
just joined
Topic Author
Posts: 13
Joined: Fri Jan 20, 2023 7:06 am

Re: RB5009UPr+S+ Bandwidth Issue

Sun Jan 29, 2023 9:27 am

To rule out the modem
Can you connect pc or so directly to modem, setting the ip manually ?
This has been tested, and was entry 2 on the troubleshooting steps taken list in my original post. The speed with that setup was as it should be, with ~940Mbps down and up.
 
holvoetn
Forum Guru
Forum Guru
Posts: 5321
Joined: Tue Apr 13, 2021 2:14 am
Location: Belgium

Re: RB5009UPr+S+ Bandwidth Issue

Sun Jan 29, 2023 10:08 am

Apologies, missed that one.

The eth port connected to the modem, what speed does it show in interface/ethernet print detail (not advertised. Speed, status tab when using winbox) ?
If 1000M is not shown, when connected to the dumb switch, does it show 1000M as speed ?
 
holvoetn
Forum Guru
Forum Guru
Posts: 5321
Joined: Tue Apr 13, 2021 2:14 am
Location: Belgium

Re: RB5009UPr+S+ Bandwidth Issue

Sun Jan 29, 2023 10:16 am

And ... already did a netinstall on that device ?
I know it comes factory default with ROS7 but it doesn't hurt to try.
 
Crabbotron
just joined
Topic Author
Posts: 13
Joined: Fri Jan 20, 2023 7:06 am

Re: RB5009UPr+S+ Bandwidth Issue

Sun Jan 29, 2023 10:21 am

The eth port connected to the modem, what speed does it show in interface/ethernet print detail (not advertised. Speed, status tab when using winbox) ?
The WAN interface (ether1, facing switch/ONT) shows 1Gbps Full Duplex.
And ... already did a netinstall on that device ?
It came with 7.6 pre-installed, I have used the built-in update utility to get 7.7 from the stable channel.
 
holvoetn
Forum Guru
Forum Guru
Posts: 5321
Joined: Tue Apr 13, 2021 2:14 am
Location: Belgium

Re: RB5009UPr+S+ Bandwidth Issue

Sun Jan 29, 2023 10:27 am

Netinstall is a bit more drastic then normal upgrade.
Give it a shot.
 
User avatar
jvanhambelgium
Forum Veteran
Forum Veteran
Posts: 985
Joined: Thu Jul 14, 2016 9:29 pm
Location: Belgium

Re: RB5009UPr+S+ Bandwidth Issue

Sun Jan 29, 2023 12:30 pm

The only test that I can think of is to disconnect the ONT/dumb-switch and effectively place a PC on your "WAN" port and "simulate" your Internet.
If you also cannot push 1Gbit/sec through the RB5009 then the unit really is faulty, really. I can't imaging a "netinstall" would magically solve such issue.
You can have a same NAT-rule in place and then transfer some files through the RB5009
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: RB5009UPr+S+ Bandwidth Issue

Sun Jan 29, 2023 6:02 pm

Did you connect ever the RB5009 to the ONT and remove the Switch and the ubiquiti to do testing??
concur your trunk bridge is not needed, only need one bridge.........and why hide private IPs in address settings, there is nothing secure about doing so ???
 
Crabbotron
just joined
Topic Author
Posts: 13
Joined: Fri Jan 20, 2023 7:06 am

Re: RB5009UPr+S+ Bandwidth Issue

Sun Jan 29, 2023 6:39 pm

Did you connect ever the RB5009 to the ONT and remove the Switch and the ubiquiti to do testing??
concur your trunk bridge is not needed, only need one bridge.........and why hide private IPs in address settings, there is nothing secure about doing so ???
Yes, that testing was accomplished, with no change. As for the IPs, I redacted my public IP. My private IPs all fall in the 10.1.0.0/16 space.
The only test that I can think of is to disconnect the ONT/dumb-switch and effectively place a PC on your "WAN" port and "simulate" your Internet.
If you also cannot push 1Gbit/sec through the RB5009 then the unit really is faulty, really. I can't imaging a "netinstall" would magically solve such issue.
You can have a same NAT-rule in place and then transfer some files through the RB5009
I will attempt this testing tonight (night shift workers unite!), and report back.


I would like to add:
I have refined my LAN testing, and iperf2 testing from my computer to a VM (both have 10G NICs) shows successful multigig service. I got 6-7Gbps, which is exactly what I expect to see there.
Also, I decided to put 7.8beta2 on the device, and my test results are now doubled. I’m now getting ~200 down and ~10 up. So I’m wondering if there might be a firmware thing going on. Unfortunately, in the 8 days my ticket has been open, MikroTik support hasn’t responded to my initial message (but I also don’t know how long it normally takes).
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: RB5009UPr+S+ Bandwidth Issue

Sun Jan 29, 2023 9:32 pm

GET RID OF ANY FUNKY DHCP Settings for now!!!
DONT MAKE A FIREWALL ADDRESS LIST WITH THE SAME NAME AS INTERFACE LIST aka LAN'
REMOVE THIS RULE FROM THE INPUT CHAIN UNTIL YOU CAN EXPLAIN THIS RULE
add action=accept chain=input dst-address=10.1.69.1 src-address=10.1.69.69

ALL your port forwarding rules are in error by missing either the dst-address (for fixed static wanip)
or in-interface-list=WAN for dynamic IP.
If you have hairpin nat (users in same subnet as server and are forced to use WANIP to access the server ) then you need to state this clearly.
DOUBLE CHECK YOUR NTP Settings, I thought it was NTP SERVER is simply ENABLED and the NTP CLient setting is where you add external servers???


# model = RB5009UPr+S+
# serial number = [redacted]
/interface bridge
add admin-mac=18:FD:74:F8:A8:0D auto-mac=no comment=defconf name=bridge vlan filtering=yes { as your last step }
/interface vlan
add interface=bridge name=vlan14 vlan-id=14
add interface=bridge name=vlan33 vlan-id=33
add interface=bridge name=vlan69 vlan-id=69
add interface=bridge name=vlan96 vlan-id=96
add interface=bridge name=homevlan vlan-id=10
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
add name=Trusted
/ip pool
add name=dhcp ranges=192.168.88.10-192.168.88.254
add name=dhcp-pool4 ranges=10.1.14.20-10.1.14.200
add name=dhcp_pool1 ranges=10.1.33.200-10.1.33.220
add name=dhcp_pool2 ranges=10.1.69.2-10.1.69.126
add name=dhcp_pool3 ranges=10.1.96.12-10.1.96.14
/ip dhcp-server
add address-pool=dhcp interface=vlanhome name=defconf
add address-pool=dhcp-pool4 interface=vlan14 name=unk-dhcp
add address-pool=dhcp_pool1 interface=vlan33 name=server-dhcp
add address-pool=dhcp_pool2 interface=vlan69 name=home-dhcp
add address-pool=dhcp_pool3 interface=vlan96 name=device-dhcp
/interface bridge port
add bridge=bridge comment=defconf interface=ether2 pvid=10 ingress filtering=yes frame-types=admit-priority-and-untagged-vlans
add bridge=bridge comment=defconf interface=ether3 pvid=10 ingress filtering=yes frame-types=admit-priority-and-untagged-vlans
add bridge=bridge comment=defconf interface=ether4 pvid=10 ingress filtering=yes frame-types=admit-priority-and-untagged-vlans
add bridge=bridge comment=defconf interface=ether5 pvid=10 ingress filtering=yes frame-types=admit-priority-and-untagged-vlans
add bridge=bridge comment=defconf interface=ether6 pvid=10 ingress filtering=yes frame-types=admit-priority-and-untagged-vlans
add bridge=bridge comment=defconf interface=ether7 pvid=10 ingress filtering=yes frame-types=admit-priority-and-untagged-vlans

add bridge=bridge interface=sfp-sfpplus1 ingress filtering=yes frame-types=admit-only-vlan-tagged
/ip neighbor discovery-settings
set discover-interface-list=Trusted
/interface bridge vlan
add bridge=bridge tagged=bridge,sfp-sfpplus1 untagged=ether2,ether3,ether4,ether5,ether6,ether7,ether8 vlan-ids=10
add bridge=bridge tagged=bridge,sfp-sfpplus1 vlan-ids=14,33,69,96

/interface list member
add comment=defconf interface=vlanhome list=LAN
add interface=vlan14 list=LAN
add interface=vlan33 list=LAN
add interface=vlan69 list=LAN
add interface=vlan96 list=LAN

add comment=defconf interface=ether1 list=WAN
add interface=ether8 list=WAN
add interface=vlanhome list=Trusted
/ip address
add address=192.168.88.1/24 comment=defconf interface=vlanhome network=\
192.168.88.0
add address=10.1.14.1/24 interface=vlan14 network=10.1.14.0
add address=10.1.33.1/24 interface=vlan33 network=10.1.33.0
add address=10.1.69.1/25 interface=vlan69 network=10.1.69.0
add address=10.1.96.1/28 interface=vlan96 network=10.1.96.0
/ip dhcp-server network
add address=192.168.88.0/24 comment=defconf dns-server=192.168.88.1 gateway=\
192.168.88.1
add address=10.1.14.0/24 dns-server=10.1.33.33 gateway=10.1.14.1
add address=10.1.33.0/24 dns-server=10.1.33.33 gateway=10.1.33.1
add address=10.1.69.0/25 dns-server=10.1.33.33 gateway=10.1.69.1
add address=10.1.96.0/28 dns-server=10.1.33.33 gateway=10.1.96.1
/ip firewall filter
add action=accept chain=input comment=\
"defconf: accept established,related,untracked" connection-state=\
established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment=\
"defconf: accept to local loopback (for CAPsMAN)" dst-address=127.0.0.1
add action=accept chain=input in-interface-list=Trusted
add action=accept chain=input in-interface-list=LAN dst-port=53,123 protocol=tcp
add action=accept chain=input in-interface-list=LAN dst-port=53 protocol=udp
add action=drop chain=input comment="drop all else" { as last rule in filter rules }

add action=accept chain=forward comment="defconf: accept in ipsec policy" \
ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" \
ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" \
connection-state=established,related hw-offload=yes
add action=accept chain=forward comment=\
"defconf: accept established,related, untracked" connection-state=\
established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" \
connection-state=invalid
add action=accept chain=forward in-interface-list=LAN out-interface-list=WAN
add action=accept chain=forward comment="port forwarding" connection-nat-state=dstnat
add actioin=drop chain=forward comment="drop all else"

/tool mac-server
set allowed-interface-list=NONE
/tool mac-server mac-winbox
set allowed-interface-list=Trusted
 
Crabbotron
just joined
Topic Author
Posts: 13
Joined: Fri Jan 20, 2023 7:06 am

Re: RB5009UPr+S+ Bandwidth Issue

Mon Jan 30, 2023 5:44 am

Okay, there's a lot to unpack here.
REMOVE THIS RULE FROM THE INPUT CHAIN UNTIL YOU CAN EXPLAIN THIS RULE
add action=accept chain=input dst-address=10.1.69.1 src-address=10.1.69.69
I was having issues with management from my computer (10.1.69.69), but anything from the 192.168.88.0/24 worked fine, so I added that rule, and haven't tried without it yet.

If you have hairpin nat (users in same subnet as server and are forced to use WANIP to access the server ) then you need to state this clearly.
I do NOT have hairpin NAT. Just segmented networks for different purposes.

ALL your port forwarding rules are in error by missing either the dst-address (for fixed static wanip)
Port forwarding is working as intended. Since there is only a single WAN IP, and I am blocking the dst-nat for certain ports, the rules are listening to all addresses.

DOUBLE CHECK YOUR NTP Settings, I thought it was NTP SERVER is simply ENABLED and the NTP CLient setting is where you add external servers???
NTP Client is active so the RB5009 can get the time, but I run an NTP server on my network for the rest of the clients.

add interface=bridge name=homevlan vlan-id=10
Why are you adding a VLAN to my network? This makes the least sense to me. I have my network segmented in a specific way, and one that is not being affected by the current issue. Speed testing from my computer (10.1.69.69) to an iperf2 server (10.1.33.4) shows that multi-gig routing is working as intended.
VLAN69 = Home Network
VLAN33 = Server Network
VLAN96 = Network Management (APs, switches, etc)
VLAN14 = IPMI for servers

add name=dhcp-pool4 ranges=10.1.14.20-10.1.14.200
VLAN14 does not need a DHCP pool/scope, as it is just for IPMI of my servers. As well, it does not need to be a /24, as there are only 3 active devices on that network (RB5009, and 2x Dell R820 iDRAC)

add bridge=bridge comment=defconf interface=ether2 pvid=10 ingress filtering=yes frame-types=admit-priority-and-untagged-vlans
add bridge=bridge comment=defconf interface=ether3 pvid=10 ingress filtering=yes frame-types=admit-priority-and-untagged-vlans
add bridge=bridge comment=defconf interface=ether4 pvid=10 ingress filtering=yes frame-types=admit-priority-and-untagged-vlans
add bridge=bridge comment=defconf interface=ether5 pvid=10 ingress filtering=yes frame-types=admit-priority-and-untagged-vlans
add bridge=bridge comment=defconf interface=ether6 pvid=10 ingress filtering=yes frame-types=admit-priority-and-untagged-vlans
add bridge=bridge comment=defconf interface=ether7 pvid=10 ingress filtering=yes frame-types=admit-priority-and-untagged-vlans
add bridge=bridge interface=sfp-sfpplus1 ingress filtering=yes frame-types=admit-only-vlan-tagged
These ports are unused, and are disabled. I don't need them in a bridge.

DONT MAKE A FIREWALL ADDRESS LIST WITH THE SAME NAME AS INTERFACE LIST aka LAN'
The list names in use are default. I didn't change the names, or add a list. This is how it came, all I did was add/remove items from the lists.

add action=accept chain=input in-interface-list=LAN dst-port=53,123 protocol=tcp
add action=accept chain=input in-interface-list=LAN dst-port=53 protocol=udp
These rules are unnecessary, as I run load-balanced DNS servers on the network, so there is a dst-nat rule going to the virtual IP (10.1.33.33).

/tool mac-server mac-winbox
set allowed-interface-list=Trusted
I don't use WinBox, since I run Linux. Since my last config upload, I have completely disable WinBox.

GET RID OF ANY FUNKY DHCP Settings for now!!!
DONT MAKE A FIREWALL ADDRESS LIST WITH THE SAME NAME AS INTERFACE LIST aka LAN'
REMOVE THIS RULE FROM THE INPUT CHAIN UNTIL YOU CAN EXPLAIN THIS RULE
I don't understand why you are so upset at me for having an issue. I had seen similar forum posts about the RB5009, but all of the fixes that worked there are not working for me, so I started my own thread. If my previous replies have come across as rude or demeaning, that is not my intention. I am just trying to get this solved, as I am paying for Gigabit internet, and would like to have access to all of that bandwidth.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: RB5009UPr+S+ Bandwidth Issue

Mon Jan 30, 2023 6:03 am

Process of elimination. Something in your setup is hindering the connectivity. My attempt is to eliminate all the extra stuff you have added.
I would also get rid of your fancy pants DNS work because perhaps that is slowing something down.
Sorry not able to pinpoint the exact cause............
No requirement to follow any of the advice though, hopefully somebody else sees something that helps!!
 
Crabbotron
just joined
Topic Author
Posts: 13
Joined: Fri Jan 20, 2023 7:06 am

Re: RB5009UPr+S+ Bandwidth Issue

Mon Jan 30, 2023 6:31 am

Process of elimination. Something in your setup is hindering the connectivity. My attempt is to eliminate all the extra stuff you have added.
I would also get rid of your fancy pants DNS work because perhaps that is slowing something down.
Sorry not able to pinpoint the exact cause............
No requirement to follow any of the advice though, hopefully somebody else sees something that helps!!
I understand the process of elimination, and see where you are coming from. I think the biggest issue I have here is that I can't seem to pinpoint the issue, despite trying many things, and trying other solutions that have worked for other people. I will say that in most of the other threads I have read, the people seem to be using a single common modem that is Comcast GPON, and they have multigig services. My ISP is not rolling out multigig until later this year (which I will be the first to get, since I work in the NOC there, and will be the guinea pig). The commonality that I have seen among many of the other posts/threads is some kind of issue when going between different line speeds. So since I have a 1G WAN and 10G LAN, that would seem to be the issue here, but testing with 1G WAN and 1G LAN also doesn't work. Testing has been done with iperf2, iperf3, Ookla speed test, and OpenSpeedTest. The iperfs and Ookla servers are hosted at my ISP switchcenter, which is about a 10 minute drive, so the fact that I cannot get good speeds with these optimal conditions is concerning.


As well, this RB5009 is here to replace my Unifi UDM-Pro, which was able to get the full 1Gbps in this exact same logical setup.


Based on everything I've read in this thread, I am basically operating under the assumption that there is either a gross overlook of a simple setting on the config (entirely possible, I've never used MikroTik routers, only switches), or there could be some firmware/hardware bug causing the issue. I'm hoping that MikroTik gets back to me soon, as I self-host many services that affect a lot of users, and the crushed upload speed has caused me to put a pause on all of that.
 
User avatar
mkx
Forum Guru
Forum Guru
Posts: 11381
Joined: Thu Mar 03, 2016 10:23 pm

Re: RB5009UPr+S+ Bandwidth Issue

Mon Jan 30, 2023 7:58 am

Just an idea: some ISPs play funky games by throttling throughput for client devices "they don't recognize". Do you may want to mimick your current router on WAN side as much as possible. Starting with MAC address. There are a few more tricks played, such as requirement for a particular QoS value used by client device even though VLAN is not used (this part, using QoS on otherwise untagged link, is a bit tricky on ROS). Etc.
 
Crabbotron
just joined
Topic Author
Posts: 13
Joined: Fri Jan 20, 2023 7:06 am

Re: RB5009UPr+S+ Bandwidth Issue

Mon Jan 30, 2023 11:29 am

Just an idea: some ISPs play funky games by throttling throughput for client devices "they don't recognize". Do you may want to mimick your current router on WAN side as much as possible. Starting with MAC address. There are a few more tricks played, such as requirement for a particular QoS value used by client device even though VLAN is not used (this part, using QoS on otherwise untagged link, is a bit tricky on ROS). Etc.
No funky games here. When a new device is put into service (if replacing an old one), all that needs to be done is to clear the ARP cache on the ISP router. No QoS values necessary, either. I have worked with my NOC colleagues, as well as our network engineering department, and they cannot seem to figure out what is going on here (but they are also Cisco-focused, go figure).

When I put this into service, I cleared the ARP cache myself, and made sure everything was good to go.

To add:
The ISP is also not playing any trickery, even unintentionally. I have a dedicated 1G active ethernet line to the house, which is fed to a non-congested 10G uplink to the ISP router (during peak times, the 10G link sees maybe 20-30% utilization). Theoretically, I could get a tech dispatched to the house, but they would end up calling me in the NOC, so that won't do much good.
 
kowal
newbie
Posts: 30
Joined: Sun Jul 06, 2014 2:23 am

Re: RB5009UPr+S+ Bandwidth Issue

Tue Jan 31, 2023 1:17 pm

Very strange issue.
I've received mine last week (non-poe version) and with 2,5G PON SFP in SFP+ port and PC with multi-gig card I'm able to push above 2Gbit over that device.
If you work in ISP NOC (like me) maybe you should try to shape that 1G connection to ex. 300,500Mb and see if it make any difference?
 
Crabbotron
just joined
Topic Author
Posts: 13
Joined: Fri Jan 20, 2023 7:06 am

Re: RB5009UPr+S+ Bandwidth Issue

Tue Jan 31, 2023 6:00 pm

Just want to post an update here. While going to reply to kowal, I decided to completely uncable the device, bring it to my office, and do some desk testing. After locking myself out due to my weird kink for security (I have already created an OOB management port to prevent this in the future), I did some poking around and some print commands. After fearing the worst (was messing with link negotiation on the SFP+ port, and locked myself out of the device), I decided to put it back in place, and go for broke. To my surprise, when I put it back in, the SFP+ port is correctly negotiating at 10G, the WAN port is negotiating at 1G, and I am now getting my full speeds. Nothing was changed, and all the status indicators (ie link speed numbers) are the same as before, but I am now getting the speeds I am supposed to be getting.

I want to thank everybody here for all of your input. I really wish I could say that I did some magic, and share it here for future generations to learn from, but I really did nothing. Or if I did do something, it was so ephemeral, my mere mortal mind cannot comprehend what it was.

Who knows, maybe physically moving it in my rack (just physical movement, no logical or cable changes) was the key. All I needed to do was move it off of my old router (which was powered off, so heat was not a factor here), so I can unrack that and sell it.

Who is online

Users browsing this forum: No registered users and 22 guests