Community discussions

MikroTik App
 
ronnielazarus
just joined
Topic Author
Posts: 16
Joined: Mon Oct 17, 2022 6:56 pm

Unable to reach Total Available Bandwidth on RB750gr3

Tue Nov 08, 2022 12:26 pm

Hi, I have a RB750gr3.

I have two WAN Uplinks with 150 and 350 mbps speeds respectively. However, after applying the '10' mangle rules, and making sure that for multi-threaded and non :443 tasks 'both addresses and ports' is used, I still can't get speeds from over ~210 mbps MAX while my CPU hits around 80 - 90 percent. I've posted similar queries on other posts of mine but they have gotten no attention and/or help around it. I've receieved vague responses such as, "RB750gr3 can't do aggregation" etc., and upon asking further, no help!

Please note that I'm new to Mikrotik but not networking in general. I've spent countless hours trying to learn the working of RouterOS and am over exceedingly happy by the same. However, the fact that my router can't fully utilize my ISP plan is a deal breaker for me. Surely I'm missing something, and would appreciate if I'm pointed in the correct direction please.

I simply need to know two things:
1. Is my PCC config the most optimal?
2. Do I need to invest in a more powerful router such as the RB5009UG+S+IN? Or is there a cheaper alternative? After having used PCC and the flexibility it offers, I don't think I can go back to basic round-robin based routers. Remember, I'm trying to hit ~600 mbps.

Below is my config:
# nov/08/2022 15:48:38 by RouterOS 7.6
# software id = AU10-A1CY
#
# model = RB750Gr3
# serial number = xxxxxxxxxxxx
/interface bridge
add name=bridge-LAN1andLAN2
/interface ethernet
set [ find default-name=ether1 ] l2mtu=1452 mac-address=F8:75:A4:AB:D1:AB mtu=1452 name=ether1-BSNL
set [ find default-name=ether2 ] comment=NetPlus name=ether2-NetPlus
set [ find default-name=ether3 ] comment="LAN - To ASUS" name=ether3-LAN1
set [ find default-name=ether4 ] comment="LAN - Free" name=ether4-LAN2
set [ find default-name=ether5 ] comment="LAN - Management" name=ether5-Management
/disk
set sd1-part1 name=disk1
/interface list
add comment="Contains all WAN Interfaces" name=WAN
add comment="Contains all LAN Interfaces" name=LAN
add comment="Contains Iterfaces with Internet" name=INTERNET
add comment="Contains all Management Interfaces" name=Management
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip pool
add name=dhcp_pool1 ranges=192.168.88.100-192.168.88.254
add name=dhcp_pool2 ranges=192.168.3.2-192.168.3.254
/ip dhcp-server
add address-pool=dhcp_pool1 interface=bridge-LAN1andLAN2 name=Bridge_DCHP_Server
add address-pool=dhcp_pool2 interface=ether5-Management name=Management_DCHP_Server
/port
set 0 name=serial0
/interface pppoe-client
add add-default-route=yes comment=BSNL disabled=no interface=ether1-BSNL max-mru=1450 max-mtu=1450 name=PPPoE-BSNL profile=default-encryption user=joxxxxxxxxxxxx.bsnl.in
/routing table
add fib name=to_ISP_BSNL
add fib name=to_ISP_NetPlus
add disabled=no fib name=only_via_BSNL
add disabled=no fib name=only_via_NetPlus
add disabled=no fib name=directly_to_ASUS
/interface bridge port
add bridge=bridge-LAN1andLAN2 interface=ether3-LAN1
add bridge=bridge-LAN1andLAN2 interface=ether4-LAN2
/ip neighbor discovery-settings
set discover-interface-list=LAN
/interface detect-internet
set detect-interface-list=all internet-interface-list=INTERNET lan-interface-list=LAN wan-interface-list=WAN
/interface list member
add interface=ether1-BSNL list=WAN
add interface=ether2-NetPlus list=WAN
add interface=ether3-LAN1 list=LAN
add interface=ether4-LAN2 list=LAN
add interface=PPPoE-BSNL list=WAN
add interface=PPPoE-BSNL list=INTERNET
add interface=ether2-NetPlus list=INTERNET
add interface=ether5-Management list=Management
add interface=bridge-LAN1andLAN2 list=LAN
/ip address
add address=192.168.88.1/24 comment=Bridge interface=bridge-LAN1andLAN2 network=192.168.88.0
add address=192.168.1.12/24 comment="To NetPlus Router" interface=ether2-NetPlus network=192.168.1.0
add address=192.168.0.10/24 comment="To BSNL Modem" interface=ether1-BSNL network=192.168.0.0
add address=192.168.3.1/24 comment=Management interface=ether5-Management network=192.168.3.0
/ip cloud
set ddns-enabled=yes
/ip dhcp-server network
add address=192.168.3.0/28 dns-server=8.8.8.8,8.8.4.4 gateway=192.168.3.1
add address=192.168.3.0/24 dns-server=8.8.8.8,8.8.4.4 gateway=192.168.3.1
add address=192.168.88.0/24 dns-server=8.8.8.8,8.8.4.4,218.248.114.129 gateway=192.168.88.1
add address=192.168.89.0/24 dns-server=8.8.8.8,8.8.4.4 gateway=192.168.89.1
/ip dns
set allow-remote-requests=yes servers=8.8.8.8,8.8.4.4
/ip firewall address-list
add address=0.0.0.0/8 comment=RFC6890 list=not_in_internet
add address=172.16.0.0/12 comment=RFC6890 list=not_in_internet
add address=192.168.0.0/16 comment=RFC6890 list=not_in_internet
add address=10.0.0.0/8 comment=RFC6890 list=not_in_internet
add address=169.254.0.0/16 comment=RFC6890 list=not_in_internet
add address=127.0.0.0/8 comment=RFC6890 list=not_in_internet
add address=224.0.0.0/4 comment=Multicast list=not_in_internet
add address=198.18.0.0/15 comment=RFC6890 list=not_in_internet
add address=192.0.0.0/24 comment=RFC6890 list=not_in_internet
add address=192.0.2.0/24 comment=RFC6890 list=not_in_internet
add address=198.51.100.0/24 comment=RFC6890 list=not_in_internet
add address=203.0.113.0/24 comment=RFC6890 list=not_in_internet
add address=100.64.0.0/10 comment=RFC6890 list=not_in_internet
add address=240.0.0.0/4 comment=RFC6890 list=not_in_internet
add address=192.88.99.0/24 comment="6to4 relay Anycast [RFC 3068]" list=not_in_internet
add address=255.255.255.255 comment=RFC6890 list=not_in_internet
add address=cloud.mikrotik.com list=mikrotik-cloud
add address=cloud2.mikrotik.com list=mikrotik-cloud
add address=whatismyipaddress.com list=what-is-my-ip
/ip firewall filter
add action=accept chain=input comment="Accept, Established & Related" connection-state=established,related
add action=drop chain=input comment="Drop Invalid Connections" connection-state=invalid
add action=accept chain=input comment="Allow ICMP" disabled=yes in-interface=PPPoE-BSNL log=yes log-prefix=FIREWALL_ICMP_HIT protocol=icmp
add action=accept chain=input comment="Allow Winbox (1964)" in-interface=PPPoE-BSNL log=yes log-prefix=FIREWALL_WINBOX_HIT port=1964 protocol=tcp
add action=accept chain=input comment="Allow SSH (1963)" dst-port=1963 in-interface=PPPoE-BSNL log=yes log-prefix=FIREWALL_SSH_HIT protocol=tcp
add action=accept chain=input comment="Allow HTTP and HTTPs (80,443)" dst-port=80,443 in-interface=PPPoE-BSNL log=yes log-prefix=FIREWALL_WebServer_HIT protocol=tcp
add action=accept chain=input comment="Allow HTTP and HTTPs (80,443)" dst-port=80,443 in-interface=PPPoE-BSNL log=yes log-prefix=FIREWALL_HTTPS_HIT protocol=udp
add action=drop chain=input comment="Block Everything Else" in-interface-list=WAN
/ip firewall mangle
add action=mark-routing chain=output dst-address-list=mikrotik-cloud log=yes new-routing-mark=only_via_BSNL passthrough=no
add action=mark-routing chain=output disabled=yes dst-address=192.168.50.0/24 log=yes log-prefix=To_ASUS new-routing-mark=directly_to_ASUS passthrough=no
add action=mark-routing chain=prerouting dst-address-list=what-is-my-ip log=yes new-routing-mark=only_via_BSNL passthrough=no
add action=accept chain=prerouting disabled=yes dst-address=192.168.0.0/24
add action=accept chain=prerouting disabled=yes dst-address=192.168.1.0/24
add action=mark-connection chain=prerouting comment="WAN to LAN" connection-mark=no-mark in-interface=PPPoE-BSNL new-connection-mark=ISP_BSNL passthrough=no
add action=mark-connection chain=prerouting connection-mark=no-mark in-interface=ether2-NetPlus new-connection-mark=ISP_NetPlus passthrough=no
add action=mark-connection chain=prerouting comment=PCC::Mangles connection-mark=no-mark dst-address-list=!not_in_internet dst-address-type=!local dst-port=80,443 in-interface-list=LAN new-connection-mark=ISP_BSNL passthrough=yes per-connection-classifier=both-addresses:10/0 \
    protocol=tcp
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-list=!not_in_internet dst-address-type=!local dst-port=80,443 in-interface-list=LAN new-connection-mark=ISP_BSNL passthrough=yes per-connection-classifier=both-addresses:10/1 protocol=tcp
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-list=!not_in_internet dst-address-type=!local dst-port=80,443 in-interface-list=LAN new-connection-mark=ISP_BSNL passthrough=yes per-connection-classifier=both-addresses:10/2 protocol=tcp
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-list=!not_in_internet dst-address-type=!local dst-port=80,443 in-interface-list=LAN new-connection-mark=ISP_NetPlus passthrough=yes per-connection-classifier=both-addresses:10/3 protocol=tcp
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-list=!not_in_internet dst-address-type=!local dst-port=80,443 in-interface-list=LAN new-connection-mark=ISP_NetPlus passthrough=yes per-connection-classifier=both-addresses:10/4 protocol=tcp
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-list=!not_in_internet dst-address-type=!local dst-port=80,443 in-interface-list=LAN new-connection-mark=ISP_NetPlus passthrough=yes per-connection-classifier=both-addresses:10/5 protocol=tcp
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-list=!not_in_internet dst-address-type=!local dst-port=80,443 in-interface-list=LAN new-connection-mark=ISP_NetPlus passthrough=yes per-connection-classifier=both-addresses:10/6 protocol=tcp
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-list=!not_in_internet dst-address-type=!local dst-port=80,443 in-interface-list=LAN new-connection-mark=ISP_NetPlus passthrough=yes per-connection-classifier=both-addresses:10/7 protocol=tcp
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-list=!not_in_internet dst-address-type=!local dst-port=80,443 in-interface-list=LAN new-connection-mark=ISP_NetPlus passthrough=yes per-connection-classifier=both-addresses:10/8 protocol=tcp
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-list=!not_in_internet dst-address-type=!local dst-port=80,443 in-interface-list=LAN new-connection-mark=ISP_NetPlus passthrough=yes per-connection-classifier=both-addresses:10/9 protocol=tcp
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-list=!not_in_internet dst-address-type=!local dst-port=80,443 in-interface-list=LAN new-connection-mark=ISP_BSNL passthrough=yes per-connection-classifier=both-addresses:10/0 protocol=udp
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-list=!not_in_internet dst-address-type=!local dst-port=80,443 in-interface-list=LAN new-connection-mark=ISP_BSNL passthrough=yes per-connection-classifier=both-addresses:10/1 protocol=udp
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-list=!not_in_internet dst-address-type=!local dst-port=80,443 in-interface-list=LAN new-connection-mark=ISP_BSNL passthrough=yes per-connection-classifier=both-addresses:10/2 protocol=udp
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-list=!not_in_internet dst-address-type=!local dst-port=80,443 in-interface-list=LAN new-connection-mark=ISP_NetPlus passthrough=yes per-connection-classifier=both-addresses:10/3 protocol=udp
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-list=!not_in_internet dst-address-type=!local dst-port=80,443 in-interface-list=LAN new-connection-mark=ISP_NetPlus passthrough=yes per-connection-classifier=both-addresses:10/4 protocol=udp
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-list=!not_in_internet dst-address-type=!local dst-port=80,443 in-interface-list=LAN new-connection-mark=ISP_NetPlus passthrough=yes per-connection-classifier=both-addresses:10/5 protocol=udp
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-list=!not_in_internet dst-address-type=!local dst-port=80,443 in-interface-list=LAN new-connection-mark=ISP_NetPlus passthrough=yes per-connection-classifier=both-addresses:10/6 protocol=udp
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-list=!not_in_internet dst-address-type=!local dst-port=80,443 in-interface-list=LAN new-connection-mark=ISP_NetPlus passthrough=yes per-connection-classifier=both-addresses:10/7 protocol=udp
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-list=!not_in_internet dst-address-type=!local dst-port=80,443 in-interface-list=LAN new-connection-mark=ISP_NetPlus passthrough=yes per-connection-classifier=both-addresses:10/8 protocol=udp
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-list=!not_in_internet dst-address-type=!local dst-port=80,443 in-interface-list=LAN new-connection-mark=ISP_NetPlus passthrough=yes per-connection-classifier=both-addresses:10/9 protocol=udp
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-list=!not_in_internet dst-address-type=!local in-interface-list=LAN new-connection-mark=ISP_BSNL passthrough=yes per-connection-classifier=both-addresses-and-ports:10/0
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-list=!not_in_internet dst-address-type=!local in-interface-list=LAN new-connection-mark=ISP_BSNL passthrough=yes per-connection-classifier=both-addresses-and-ports:10/1
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-list=!not_in_internet dst-address-type=!local in-interface-list=LAN new-connection-mark=ISP_BSNL passthrough=yes per-connection-classifier=both-addresses-and-ports:10/2
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-list=!not_in_internet dst-address-type=!local in-interface-list=LAN new-connection-mark=ISP_NetPlus passthrough=yes per-connection-classifier=both-addresses-and-ports:10/3
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-list=!not_in_internet dst-address-type=!local in-interface-list=LAN new-connection-mark=ISP_NetPlus passthrough=yes per-connection-classifier=both-addresses-and-ports:10/4
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-list=!not_in_internet dst-address-type=!local in-interface-list=LAN new-connection-mark=ISP_NetPlus passthrough=yes per-connection-classifier=both-addresses-and-ports:10/5
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-list=!not_in_internet dst-address-type=!local in-interface-list=LAN new-connection-mark=ISP_NetPlus passthrough=yes per-connection-classifier=both-addresses-and-ports:10/6
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-list=!not_in_internet dst-address-type=!local in-interface-list=LAN new-connection-mark=ISP_NetPlus passthrough=yes per-connection-classifier=both-addresses-and-ports:10/7
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-list=!not_in_internet dst-address-type=!local in-interface-list=LAN new-connection-mark=ISP_NetPlus passthrough=yes per-connection-classifier=both-addresses-and-ports:10/8
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-list=!not_in_internet dst-address-type=!local in-interface-list=LAN new-connection-mark=ISP_NetPlus passthrough=yes per-connection-classifier=both-addresses-and-ports:10/9
add action=mark-routing chain=prerouting comment="Outgoing Traffic with PCC" connection-mark=ISP_BSNL in-interface-list=LAN new-routing-mark=to_ISP_BSNL passthrough=no
add action=mark-routing chain=prerouting connection-mark=ISP_NetPlus in-interface-list=LAN new-routing-mark=to_ISP_NetPlus passthrough=no
add action=mark-routing chain=output connection-mark=ISP_BSNL new-routing-mark=to_ISP_BSNL out-interface=PPPoE-BSNL passthrough=no
add action=mark-routing chain=output connection-mark=ISP_NetPlus new-routing-mark=to_ISP_NetPlus out-interface=ether2-NetPlus passthrough=no
/ip firewall nat
add action=dst-nat chain=dstnat comment="NGINX Reverse Proxy Forward to ASUS" dst-port=80 in-interface=PPPoE-BSNL log=yes log-prefix=HIT_NON_Secure_WebServer_NGINX port="" protocol=tcp to-addresses=192.168.88.2 to-ports=81
add action=dst-nat chain=dstnat dst-port=443 in-interface=PPPoE-BSNL log=yes log-prefix=HIT_Secure_WebServer_NGINX port="" protocol=tcp to-addresses=192.168.88.2 to-ports=444
add action=dst-nat chain=dstnat dst-port=80,433 in-interface=PPPoE-BSNL log=yes log-prefix=HIT_WebServer_NGINX port="" protocol=udp to-addresses=192.168.88.2
add action=dst-nat chain=dstnat comment="OpenVPN Forward to ASUS" dst-port=1194 in-interface=PPPoE-BSNL log=yes log-prefix=OpenVPN protocol=udp to-addresses=192.168.88.2 to-ports=1194
add action=masquerade chain=srcnat comment="General NAT for all WAN (includes Failover)" out-interface=PPPoE-BSNL
add action=masquerade chain=srcnat out-interface=ether2-NetPlus protocol=tcp to-addresses=192.168.1.1
/ip firewall service-port
set sip disabled=yes
/ip route
add check-gateway=ping comment="Default Route for NetPlus" disabled=no distance=1 dst-address=0.0.0.0/0 gateway=192.168.1.1 routing-table=main scope=30 suppress-hw-offload=no target-scope=10
add comment="Load Balancing Route to ISP BSNL" disabled=no distance=1 dst-address=0.0.0.0/0 gateway=PPPoE-BSNL pref-src="" routing-table=to_ISP_BSNL scope=30 suppress-hw-offload=no target-scope=10
add check-gateway=ping comment="Load Balancing Route to ISP NetPlus" disabled=no distance=1 dst-address=0.0.0.0/0 gateway=192.168.1.1@main pref-src="" routing-table=to_ISP_NetPlus scope=30 suppress-hw-offload=no target-scope=10
add comment="For Cloud Update" disabled=no distance=1 dst-address=0.0.0.0/0 gateway=PPPoE-BSNL pref-src="" routing-table=only_via_BSNL scope=30 suppress-hw-offload=no target-scope=10
add check-gateway=ping comment="Only via NetPlus" disabled=no distance=1 dst-address=0.0.0.0/0 gateway=192.168.1.1 pref-src="" routing-table=only_via_NetPlus scope=30 suppress-hw-offload=no target-scope=10
add comment="Default Route for BSNL" disabled=no distance=2 dst-address=0.0.0.0/0 gateway=PPPoE-BSNL pref-src="" routing-table=main scope=30 suppress-hw-offload=no target-scope=10
add disabled=no distance=1 dst-address=192.168.50.0/24 gateway=192.168.88.2 pref-src="" routing-table=main scope=30 suppress-hw-offload=no target-scope=10 vrf-interface=ether3-LAN1
/ip service
set telnet disabled=yes
set ftp disabled=yes
set www disabled=yes
set ssh port=1963
set api disabled=yes
set winbox port=1964
set api-ssl disabled=yes
/system clock
set time-zone-autodetect=no time-zone-name=Asia/Kolkata
/system scheduler
add interval=20m name=Update_Google_Domains_IP on-event="# Variables\r\
    \n# Username and password are the long cryptic ones Google Domains provides,\r\
    \n# not your regular Google account ones\r\
    \n:local GoogleDNSUsername \"xxxxxxxxxx\"\r\
    \n:local GoogleDNSPassword \"xxxxxxxxx\"\r\
    \n:local hostName \"xxxxxxxxxx\"\r\
    \n:local currentIP \"\"\r\
    \n:local setResults \"\"\r\
    \n:local previousIP \"\"\r\
    \n\r\
    \n# Script\r\
    \n:set currentIP [/ip cloud get public-address]\r\
    \n:set previousIP [:resolve \"\$hostName\"]\r\
    \n\r\
    \n:if (\$currentIP != \$previousIP) do={\r\
    \n:do {\r\
    \n/tool fetch url=\"https://\$GoogleDNSUsername:\$GoogleDNSPassword@domains.google.com/nic/update\?hostname=\$hostName&myip=\$currentIP\" mode=https dst-path=/disk1/GoogleDNS.txt\r\
    \n:set setResults [/file/ get disk1/GoogleDNS.txt contents];\r\
    \n:log info (\"GoogleDNS said this: \$setResults\")\r\
    \n} on-error={ \r\
    \n:log error (\"GoogleDNS: script failed to set new IP address\") \r\
    \n}\r\
    \n}" policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon start-date=oct/20/2020 start-time=22:55:00
add interval=1d name="Reboot Every Morning @ 4:30 AM" on-event="/system reboot" policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon start-date=oct/20/2020 start-time=04:30:00
/tool bandwidth-server
set enabled=no
/tool mac-server
set allowed-interface-list=LAN
/tool sniffer
set file-limit=500000KiB file-name=/disk1/ron.pcap filter-interface=ether1-BSNL,PPPoE-BSNL,ether3-LAN1 streaming-server=192.168.50.20


And below is a high-level network topology I have.
Network_Diagram-Page-2.drawio.png
Help and guidance is much appreciated.

Kind Regards,
You do not have the required permissions to view the files attached to this post.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18959
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: Unable to reach Total Available Bandwidth on RB750gr3

Tue Nov 08, 2022 2:29 pm

Cannot help a person who doesnt see the facts staring right at them.
You want 600Mbps but you only have a wan of 150 and a wan of 300, the best you can do is some percentage of 300, maybe 280-290 etc......

The hex is perfectly suited for those WAN speeds.....
 
ronnielazarus
just joined
Topic Author
Posts: 16
Joined: Mon Oct 17, 2022 6:56 pm

Re: Unable to reach Total Available Bandwidth on RB750gr3

Tue Nov 08, 2022 3:13 pm

Would especially be surprised to get any help from a person who has hastily read things incorrectly and deliberately chosen to have an insulting demeanour. I worry if the mods on this community do anything to discourage this kind of passive toxicity or if it's just become a norm and politeness is a thing of the past. :)

Either ways, my WANs are are 150 and '350', not 300. Secondly, I'm usually able to get give or take ~50mbps more than what both of my fibre plans dictate on the paper, which btw is not a third world scenario but very much a common thing. With that said, combining 200 and 400 actually results in 600 mbps which I was able to easily achieve on my ASUS RT-AX88U. Well the ASUS could even hit 650 on sites such as fast.com but that's not the point. My baseline value is at least 600. Sometimes when one or more of the uplinks open up a bit too much I wouldn't object to an even higher bandwidth than 600 obviously.

My question simply lies in the fact that when other devices were able to do it, why couldn't the hEX. My not-so-extended experience with Mikrotik suggests that either a hardware cap or software. My request from this comminuty is to:
1. Help me fix/optimize my config so that I could get over the software cap if I've by mistake not set up something correctly.
2. If the hEX isn't capable of such speeds with PCC, I'm happy in investing into something more powerful, but would appreciate guidance on if that would be actually necessary. The reason why I'm even thinking that it could be a hardware bottleneck is because of hard evidence of my CPU reaching 90 percent at times.

For example, here's a recent speedtest I did:
Screenshot 2022-11-08 183926.png
And here is the util while that was going on, right till the end:
Screenshot 2022-11-08 183902.png
You do not have the required permissions to view the files attached to this post.
Last edited by BartoszP on Tue Nov 08, 2022 3:39 pm, edited 1 time in total.
Reason: removed excessive quotting of preceding post; be wise, quote smart.
 
biomesh
Long time Member
Long time Member
Posts: 561
Joined: Fri Feb 10, 2012 8:25 pm

Re: Unable to reach Total Available Bandwidth on RB750gr3

Tue Nov 08, 2022 3:27 pm

I'm pretty sure you are hitting the max that this device can provide, considering you are using PPPOE and multi-wan (with the associated rules). The performance tests for 6.x show for a 512 packet size of 385.4 Mbps speed, which will be higher than 7.x due to the removal of the Linux route cache in the newer linux kernel.

A hap ac2, 4011, 5009 would be good choices - with the later two giving you more performance "breathing room".
 
holvoetn
Forum Guru
Forum Guru
Posts: 5317
Joined: Tue Apr 13, 2021 2:14 am
Location: Belgium

Re: Unable to reach Total Available Bandwidth on RB750gr3

Tue Nov 08, 2022 3:32 pm

Getting more then you expect, that's a new one for me... I wish my ISP would do the same.

I think your problem might also be with the PCC setup.
Only 1 WAN itf can be used at a time per stream if I am not mistaking.

What total bandwidth do you get with multiple PCs downloading ?
 
User avatar
Znevna
Forum Guru
Forum Guru
Posts: 1347
Joined: Mon Sep 23, 2019 1:04 pm

Re: Unable to reach Total Available Bandwidth on RB750gr3

Tue Nov 08, 2022 4:01 pm

Short version: without fasttrack and with so many firewall rules you can't ask for more from that CPU.
You need more MHz.
And no, hAP ac2 won't be a better alternative.
LE: your AX88U has a quad core ARM CPU running at 1.8GHz, your RB750Gr3 has a dual core / 4 threads MIPS CPU running at 880MHz, it's a not a fair comparison that you did above.
 
ronnielazarus
just joined
Topic Author
Posts: 16
Joined: Mon Oct 17, 2022 6:56 pm

Re: Unable to reach Total Available Bandwidth on RB750gr3

Tue Nov 08, 2022 4:24 pm

I think your problem might also be with the PCC setup.
Only 1 WAN itf can be used at a time per stream if I am not mistaking.
You're mostly correct, as long as the endpoint you're connected can only do a single thread! My examples and tests are all multi-threaded however.
What total bandwidth do you get with multiple PCs downloading ?
Near about the same as the speedtest result above as long as the tik is in the picture.
 
ronnielazarus
just joined
Topic Author
Posts: 16
Joined: Mon Oct 17, 2022 6:56 pm

Re: Unable to reach Total Available Bandwidth on RB750gr3

Tue Nov 08, 2022 4:36 pm

LE: your AX88U has a quad core ARM CPU running at 1.8GHz, your RB750Gr3 has a dual core / 4 threads MIPS CPU running at 880MHz, it's a not a fair comparison that you did above.
That comparison above was simply to verify the trajectory of reason, not to put down one device or another. :) Believe this has been concluded now that the 'real' reason why I'm bottlenecking is because of the humble CPU in the rb750gr3 and the CPU alone?

Because it is true that when one of my WAN uplinks is completely down, I do get the full bandwidth of the WAN which is active at that moment be it the 150 meg one or the one having 350.

If so, are you sure that with my existing rules and the available WAN speeds in place, and also taking into account some highly appreciated 'breathing room' as @biomesh pointed out, is my only choice the 5009? And even so, with this many rules in existence, will even the CPU in 5009 be able to properly support the WAN Load? As far my calculations go it should. Sorry for the noob question though but I just need to be sure as it is indeed an expensive investment.

Kind Regards,
 
holvoetn
Forum Guru
Forum Guru
Posts: 5317
Joined: Tue Apr 13, 2021 2:14 am
Location: Belgium

Re: Unable to reach Total Available Bandwidth on RB750gr3

Tue Nov 08, 2022 4:39 pm

Multi-thread or not, you're still going with same source and end-address according to your PCC settings so 1 WAN itf.
And since you're behind NAT on that ASUS, that will always be the case.

Do the test again with multiple PCs but keep an eye on the itf overview on Hex. I think you will only see 1 itf being used.
 
ronnielazarus
just joined
Topic Author
Posts: 16
Joined: Mon Oct 17, 2022 6:56 pm

Re: Unable to reach Total Available Bandwidth on RB750gr3

Tue Nov 08, 2022 5:37 pm

Multi-thread or not, you're still going with same source and end-address according to your PCC settings so 1 WAN itf.
And since you're behind NAT on that ASUS, that will always be the case.

Do the test again with multiple PCs but keep an eye on the itf overview on Hex. I think you will only see 1 itf being used.
I see what you mean. Since the tik sees the ASUS as the source. And that makes sense. However I have tried to set up my PCC in a way that this behaviour is only permissible for ports 80, 443 and the likes, so that HTTPs doesn't break.

Like you said, I have a lot of rules, and my intention with those, specifically the last 10 rules with both address and ports in the config, was to further randomize the WAN selection so that multi-threaded applications, which do not specifically rely on ports 443 and the likes, can get data traversed through multiple WANs.

To give an example of this, I have just now done a test with a PC behind the ASUS, PFB the results, viz BSNL-150, NetPlus-350:
https://drive.google.com/file/d/1Rh4W-p ... share_link (sorry for the drive link as I'm unable to attach a 2.8mb gif directly)

You could see above that both my uplinks are under load when I do a speedtest on fast.com for example. A similar scenario takes place when I'm downloading an ISO from microsoft, or a random CDN.

I could be blatantly wrong in my understanding of what's actually happening as well. That's where I'd need your help and guidance though.

Thanks again,
Kind Regards,
 
User avatar
Znevna
Forum Guru
Forum Guru
Posts: 1347
Joined: Mon Sep 23, 2019 1:04 pm

Re: Unable to reach Total Available Bandwidth on RB750gr3

Tue Nov 08, 2022 5:38 pm

Boy, I didn't even look that further.
Looking just at your PCC rules, you're doing some weird stuff there in my opinion.
Regarding what @holvoetn is trying to say, your PCC rules have either both-addresses or both-addresses-and-ports, but your source address is always the same, 192.168.88.2, since everything sits behind that Asus.
I'm surprised you don't have general internet usage problems, https etc.
You'll have to redo your PCC rules and rethink your network (don't do another NAT on the Asus).
What was wrong using just the Asus? Curious.
 
ronnielazarus
just joined
Topic Author
Posts: 16
Joined: Mon Oct 17, 2022 6:56 pm

Re: Unable to reach Total Available Bandwidth on RB750gr3

Tue Nov 08, 2022 6:02 pm

Boy, I didn't even look that further.
Looking just at your PCC rules, you're doing some weird stuff there in my opinion.
Regarding what @holvoetn is trying to say, your PCC rules have either both-addresses or both-addresses-and-ports, but your source address is always the same, 192.168.88.2, since everything sits behind that Asus.
Well since I'm new, it is highly possible that even after rigorous RnD I have managed to put in something 'weird', even though I thought that it wasn't the case. Would you be able to very specifically target a couple of those wirdness'es so that it helps me better understand my mistakes? Thanks.

Regarding what @holvoetn is trying to say, your PCC rules have either both-addresses or both-addresses-and-ports, but your source address is always the same, 192.168.88.2, since everything sits behind that Asus.
Yes, that's perfectly clear, thanks. And like I said in my previous post, I believe the conifguration should still work as expected. Should it not?

What was wrong using just the Asus? Curious.
Simply the fact that it can't do anything similar to PCC and mostly everything using HTTPs which is almost everything now a days, just breaks! I mean on my XBOX the page that comes before each game opens to sync the game data used to stay there for 2 minutes (ideally it takes 2 seconds) and finally fail and would then work on the second or the third attempt. With the ASUS, everything is simply Round Robin. Though if that's what you want, it works perfectly.

You'll have to redo your PCC rules and rethink your network (don't do another NAT on the Asus).
I agree with not doing another NAT on the ASUS. By I need an end-user solution that's easier to manage by the family as well. Plus the Monitoring, Adaptive QoS, AiMesh, Ease of connectivity etc. is much refined in the ASUSWRT for the general polulation than it is in RouterOS. Hence, having the ASUS as a termination point for the clients is kind of unavaoidable.

I'm surprised you don't have general internet usage problems, https etc.
Whatever do you mean? So far things are working perfectly fine and I'm yet to face a general internet usage problem though. With ofcourse the obvious exception that I'm not getting the full bandwidth of both my WAN uplinks. As I said in my original post, I have put in rules specifically to ensure that HTTPs doesn't break. As general purpose multi-threaded downloads/uploads and browsing banking websites can be defined as two different things! Am I missing something here?

The way it all started and the reason why I even got the tik and am even considering investing in a more powerful one is to have the flexibility that it offers in terms of the load balancing algo. My ASUS wasn't able to loadblance as needed and that was a deal breaker, while it did everything else like a champ! I was hoping that simply adding a cost-effective loadbalancer would mitigate the ASUS's limitation. With WFH being the new black, I'm unable to survive without the kind of HA it offers, as I'm sure most of us!
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18959
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: Unable to reach Total Available Bandwidth on RB750gr3

Tue Nov 08, 2022 10:27 pm

 
ronnielazarus
just joined
Topic Author
Posts: 16
Joined: Mon Oct 17, 2022 6:56 pm

Re: Unable to reach Total Available Bandwidth on RB750gr3

Wed Nov 09, 2022 10:16 am

Thanks for copy/pasting those. However, please accept my apologies in advance and I don't mean to be rude but I feel like I need to put it out there, that if you have a problem with this post, or if you're feeling rather unhelpful today, request you to please no longer comment/respond with vague and sometimes wrong responses. Please, I beg of you to not do that! This just creates a lot of confusion and is unnecessary and unhelpful. If you were simply trying to help however and didn't realize that I would've already had those PPTs, then my sincere apologies and many thanks kind sir.

And my PCC config is perfectly inline with those presentations as they are ones coupled with @sindy's great responses (especially the PCC calculation one) all over the forum which I've used as my baseline!

I'm also currently 90% done in creating an easy to use PCC rules calculator for upto 8 WANs for folks who want a quick calculation done. I have written it in vanilla JS and some jQuery so it's easy to understand as well. Would like to give back to this community as much as I can as RouterOS is really something that I appreciate!
 
User avatar
Znevna
Forum Guru
Forum Guru
Posts: 1347
Joined: Mon Sep 23, 2019 1:04 pm

Re: Unable to reach Total Available Bandwidth on RB750gr3

Wed Nov 09, 2022 10:42 am

You insist too much on those PCC rules, and judging by your gif posted earlier you only managed to get under 10 Mbps (more like ~5Mbps) from one ISP and about 240 Mbps from the other.
Those 5-10 Mbps don't justify the complexity you added to your firewall.
Regarding your "fix" for https and banking websites, you insist too much on it, it's just a 3:7 split, nothing fancy, things might break, just no so often as it would with a 5:5 split, you can call it luck at this point.
One thing that will surely break is something like Webex, that uses a few IPs to communicate and it expects connections from the same IP that you logged in (and not necessarily using port 80 or 443), if somehow something ends up on the other ISP you'll be screaming at Cisco or whatever broke, because it can't handle your "https and banking proof" PCC.
If you're only hosting linux distros distributing them via torrent, yeah, that doesn't care much about IPs. Anything else works just by luck.
A more sane approach to split the bandwidth would be by src-address (ie. some clients go via one ISP, some via another, and you can fine tune some devices to go via either one, or add some extra rules un top of that). But that seems impossible in your setup because you insist on doing another NAT level via that Asus for some reason. (Why does anyone else need to mess with the Asus? You set it up and that's it. What could others be doing all day on it?)
And like said previously, you need a more beefy CPU.
Cheers.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11967
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Unable to reach Total Available Bandwidth on RB750gr3

Wed Nov 09, 2022 10:55 am

[…] Thanks for copy/pasting those […]
How nice, sometimes there is someone worse than me.
 
ronnielazarus
just joined
Topic Author
Posts: 16
Joined: Mon Oct 17, 2022 6:56 pm

Re: Unable to reach Total Available Bandwidth on RB750gr3

Wed Nov 09, 2022 11:24 am

One thing that will surely break is something like Webex, that uses a few IPs to communicate and it expects connections from the same IP that you logged in (and not necessarily using port 80 or 443), if somehow something ends up on the other ISP you'll be screaming at Cisco or whatever broke, because it can't handle your "https and banking proof" PCC.
You're absolutely correct! Webex Voip does break every once in a while, well mostly, I did notice that, and decided to live with it not knowing much what else could have I done there..
A more sane approach to split the bandwidth would be by src-address (ie. some clients go via one ISP, some via another, and you can fine tune some devices to go via either one, or add some extra rules un top of that). But that seems impossible in your setup because you insist on doing another NAT level via that Asus for some reason. (Why does anyone else need to mess with the Asus? You set it up and that's it. What could others be doing all day on it?)
Thanks for this, that makes sense. However I feel like I've been misguided by a lot many other posts on the forum that I traversed through.
And on that topic, reasons behind using the ASUS are all mentioned above. The lilteral most important one is a feature called 'Adaptive QoS', which doesn't just limit based upon the max available bandwidth but does something similar to DPI (but not at all quite like cz HTTPs) based upon Trenmicro's engine and prioritizes packets accordingly. Its awfully simple to use and works even with the clients behind the ASUS.
And like said previously, you need a more beefy CPU.
So provided I get a beefier CPU, is there no way other than making the new beefier tik the center? If I go with that, thereby making both the ASUS devices dumb APs, I would certainly be loosing out on Adaptive QoS and would have to rely on the tik's queing mechanism (not saying that it's impossible or not good enough). I guess I'm simply hesitant psychologically to not fully utilize my already made investment on the ASUS (this was before I was exposed to Mikrotik). But I'm ready to admit when I'm wrong and can try to go the correct route if it is true that I can no longer use it the way I want in regards to the current requirements. On top of that, I would have to set up individual rules for making one client go through one interface, but if I do that, how would I achieve a combination of that client being able to safely surf HTTPs while getting combined bandwidth on the rest? Since I have a lot of clients with that requirement, wouldn't my rules be exceedingly large in number?

Regards,
 
ronnielazarus
just joined
Topic Author
Posts: 16
Joined: Mon Oct 17, 2022 6:56 pm

Re: Unable to reach Total Available Bandwidth on RB750gr3

Wed Nov 09, 2022 4:24 pm

You insist too much on those PCC rules, and judging by your gif posted earlier you only managed to get under 10 Mbps (more like ~5Mbps) from one ISP and about 240 Mbps from the other.
Those 5-10 Mbps don't justify the complexity you added to your firewall.
Also, something tells me that the inability to get over the seen 5 - 10 mbps is simply because of the hardware limitation on the tik? PCC and all it's complexity asides, should I not be able to get around my desired 550~600 plus better utilization under both ISP simply if my hardware was upto the mark? Or is it not as straighforward as that..?
 
holvoetn
Forum Guru
Forum Guru
Posts: 5317
Joined: Tue Apr 13, 2021 2:14 am
Location: Belgium

Re: Unable to reach Total Available Bandwidth on RB750gr3

Wed Nov 09, 2022 4:33 pm

Beefier hardware and problematic configuration or settings = still problems.
 
ronnielazarus
just joined
Topic Author
Posts: 16
Joined: Mon Oct 17, 2022 6:56 pm

Re: Unable to reach Total Available Bandwidth on RB750gr3

Wed Nov 09, 2022 5:33 pm

Beefier hardware and problematic configuration or settings = still problems.
That's a given! However like I said, it is not an easy feat for me to remove the ASUS from the picture. Some of the things it is doing the tik can't do. My intention was to simply get a loadbalancer in front of the ASUS. With that said, what can or should I do to achieve that? Or is just not achievable!? Should I use a different algo?
 
dave3
newbie
Posts: 45
Joined: Mon Feb 07, 2022 8:06 am

Re: Unable to reach Total Available Bandwidth on RB750gr3

Wed Nov 09, 2022 6:52 pm

Something you could consider, on systems that are sensitive to IP changes, would be to use wireguard (perhaps setup on a good/reliable vps with a static IP). Wireguard doesn't care what IP address the packets come from, as long as the key is correct.

I've tested using a SIP app on my cellphone, connected to wireguard. With both wifi and mobile data enabled, I can turn off the wifi, and it switches to the mobile data, with a different IP, and the SIP/voip audio doesn't drop out and it keeps going like nothing happened.

Also, I don't think most secure/https websites care much if your ip address changes. There may be a few special ones that do (you could use wireguard or a vpn for those). But I sometimes turn a vpn on while I'm on an secure/https website, and don't notice any issue. Youtube/google once asked me to verify my password, but now it seems to recognize the other IP without issue.
 
ronnielazarus
just joined
Topic Author
Posts: 16
Joined: Mon Oct 17, 2022 6:56 pm

Re: Unable to reach Total Available Bandwidth on RB750gr3

Thu Nov 10, 2022 12:53 pm

Something you could consider, on systems that are sensitive to IP changes..
Many thanks Dave for your kind adivse. Yes, that would definitely work. But Wireguard in itself is not that linient on the CPU, considering I'm anyways using an underpowered hardware, this might not work in my current scenario. Thankfully, Webex Voip is not something that I'm heavily reliant on. Plus is does work most of times thankfully.

Cheers,

Who is online

Users browsing this forum: Google [Bot], lurker888 and 62 guests