measured WAN bandwidth + dynamic queue calculation script

dash · Wed Nov 30, 2016 2:09 pm

Hi guys,
as other people I often have problems with my Queue Tree settings when the internet provider is not able to deliver the full bandwidth. On my 100Mbit fiber it varies between 100Mbit and 50Mbit. In the case you tuned your Queue to a 100Mbit max-limit but only reach 50Mbit (because of the provider) the Queue will not work at all. A single download could make the internet connection unusable for others. Because of that I decided to build a bandwidth dependent Queue setting script. My script measures the WAN UP and DOWN limitis and sets Queue Tree values (percentual calculated) accordingly.

PLEASE NOTE: that it will need a lot of modifications to work with your Queue Tree config (see picture below) - it is just intended as a 'proposal' or food for though for other users...

#set parent queue variables
:local parentqueue1 "DOWNLOAD"
:local parentqueue2 "UPLOAD"
:local parentqueue3 "4 ADMIN-down"
:local parentqueue4 "4 ADMIN-up"
:local parentqueue5 "1 ALL-down"
:local parentqueue6 "1 ALL-up"
:local parentqueue7 "8 GAST-down"
:local parentqueue8 "8 GAST-up"

#clearing DOWN queue limit-at and reduce max-limit values
:local maxD 500k
:local maxU 250k
/queue tree set "$parentqueue1" limit-at=0
/queue tree set "$parentqueue1" max-limit=$maxD
/queue tree set "$parentqueue3" limit-at=0
/queue tree set "$parentqueue3" max-limit=$maxD
/queue tree set "$parentqueue5" limit-at=0
/queue tree set "$parentqueue5" max-limit=$maxD
:foreach i in=[/queue tree find parent="$parentqueue3"] do={
	:local name [/queue tree get $i name]
	/queue tree set "$name" limit-at=0
	/queue tree set "$name" max-limit=$maxD
}
#clearing UP queue limit-at and reduce max-limit values
/queue tree set "$parentqueue2" limit-at=0
/queue tree set "$parentqueue2" max-limit=$maxU
/queue tree set "$parentqueue4" limit-at=0
/queue tree set "$parentqueue4" max-limit=$maxU
/queue tree set "$parentqueue6" limit-at=0
/queue tree set "$parentqueue6" max-limit=$maxU
:foreach i in=[/queue tree find parent="$parentqueue4"] do={
	:local name [/queue tree get $i name]
	/queue tree set "$name" limit-at=0
	/queue tree set "$name" max-limit=$maxU
}

#bandwidth determination
:global "speedtest-server"
:local txAvg
:local rxAvg
:local UPpercent 95
:local DOWNpercent 80
#checking if speedtest-server is up
:if ($"speedtest-server" != "down") do={
	#measuring tx rx bandwidth
	/tool bandwidth-test 207.32.195.2 duration=20s protocol=tcp direction=both user=btest password=btest do={
		:if ($txAvg < $"tx-10-second-average") do={
			:set txAvg $"tx-10-second-average"
		}
		:if ($rxAvg < $"rx-10-second-average") do={
			:set rxAvg $"rx-10-second-average"
		}
	}
} else={
#set static bandwidth values
:set txAvg 9500000
:set rxAvg 85500000
}

#set environment variables
#staticSum is the sum of static queue values (messaging,winbox,quake)
:local staticSumD (750000 + 250000 + 250000)
:local staticSumU (750000 + 250000 + 250000)
#set base value for percentual bandwidth calculations
:local admbaseU ((($txAvg / 100) * $UPpercent) - 1500000 - 1300000)
:local admbaseUk (($admbaseU / 1000). "k")
:local admbaseU ($admbaseU - $staticSumU)
:local admbaseD ((($rxAvg / 100) * $DOWNpercent) - 1500000 - 12500000)
:local admbaseDk (($admbaseD / 1000). "k")
:local admbaseD ($admbaseD - $staticSumD)
#converting measured tx and rx to integer Mbit
:set $txAvgk ((:tonum (($txAvg / 100) * $UPpercent) / 1000) . "k")
:set $rxAvgk ((:tonum (($rxAvg / 100) * $DOWNpercent) / 1000) . "k")
:set $"txAvgk-half" ((:tonum ((($txAvg / 100) * $UPpercent) / 1000) / 2) . "k")
:set $"rxAvgk-half" ((:tonum ((($rxAvg / 100) * $DOWNpercent) / 1000) / 2) . "k")
#calculating percent values for the dynamic queue settings
:local 10percentDk ((($admbaseD / 100 * 10) / 1000) . "k")
:local 15percentDk ((($admbaseD / 100 * 15) / 1000) . "k")
:local 20percentDk ((($admbaseD / 100 * 20) / 1000) . "k")
:local 25percentDk ((($admbaseD / 100 * 25) / 1000) . "k")
:local 30percentDk ((($admbaseD / 100 * 30) / 1000) . "k")
:local 35percentDk ((($admbaseD / 100 * 35) / 1000) . "k")
:local 40percentDk ((($admbaseD / 100 * 40) / 1000) . "k")
:local 45percentDk ((($admbaseD / 100 * 45) / 1000) . "k")
:local 10percentUk ((($admbaseU / 100 * 10) / 1000) . "k")
:local 15percentUk ((($admbaseU / 100 * 15) / 1000) . "k")
:local 20percentUk ((($admbaseU / 100 * 20) / 1000) . "k")
:local 25percentUk ((($admbaseU / 100 * 25) / 1000) . "k")
:local 30percentUk ((($admbaseU / 100 * 30) / 1000) . "k")
:local 35percentUk ((($admbaseU / 100 * 35) / 1000) . "k")
:local 40percentUk ((($admbaseU / 100 * 40) / 1000) . "k")
:local 45percentUk ((($admbaseU / 100 * 45) / 1000) . "k")

#set DOWN max-limit to measured value
/queue tree set "$parentqueue1" max-limit=$rxAvgk
/queue tree set "$parentqueue3" max-limit=$rxAvgk
/queue tree set "$parentqueue5" max-limit=$rxAvgk
/queue tree set "$parentqueue7" max-limit=$"rxAvgk-half"
:foreach i in=[/queue tree find parent="$parentqueue3"] do={
	:local name [/queue tree get $i name]
	/queue tree set "$name" max-limit=$rxAvgk
}
:foreach i in=[/queue tree find parent="$parentqueue7"] do={
	:local name [/queue tree get $i name]
	/queue tree set "$name" max-limit=$"rxAvgk-half"
}
#set UP max-limit to measured value
/queue tree set "$parentqueue2" max-limit=$txAvgk
/queue tree set "$parentqueue4" max-limit=$txAvgk
/queue tree set "$parentqueue6" max-limit=$txAvgk
/queue tree set "$parentqueue8" max-limit=$"txAvgk-half"
:foreach i in=[/queue tree find parent="$parentqueue4"] do={
	:local name [/queue tree get $i name]
	/queue tree set "$name" max-limit=$txAvgk
}
:foreach i in=[/queue tree find parent="$parentqueue8"] do={
	:local name [/queue tree get $i name]
	/queue tree set "$name" max-limit=$"txAvgk-half"
}

#set limit-at for static queues without percentual calculation
/queue tree set "1 winbox-adm-down" limit-at=250k
/queue tree set "2 quake-adm-down" limit-at=250k
/queue tree set "6 messaging-adm-down" limit-at=750k
/queue tree set "$parentqueue5" limit-at=1500k
/queue tree set "1 winbox-adm-up" limit-at=250k
/queue tree set "2 quake-adm-up" limit-at=250k
/queue tree set "6 messaging-adm-up" limit-at=750k
/queue tree set "$parentqueue6" limit-at=1500k

#set 10% bandwidth queues
/queue tree set "4 streaming-adm-up" limit-at=$10percentUk

#set 20% bandwidth queues
/queue tree set "4 streaming-adm-down" limit-at=$20percentDk
/queue tree set "7 http-adm-up" limit-at=$20percentUk
/queue tree set "8 other-adm-down" limit-at=$20percentDk

#set 25% bandwidth queues
/queue tree set "7 http-adm-down" limit-at=$25percentDk
/queue tree set "8 other-adm-up" limit-at=$25percentUk

#set 35% bandwidth queues
/queue tree set "5 synosrv-adm-down" limit-at=$35percentDk

#set 45% bandwidth queues
/queue tree set "5 synosrv-adm-up" limit-at=$45percentUk

#set ADMIN-down/up limit-at value
/queue tree set "4 ADMIN-down" limit-at=$admbaseDk
/queue tree set "4 ADMIN-up" limit-at=$admbaseUk

Please see the attached picture to see my Queue tree config for that the script was build for

queue_tree_example.PNG

dash · Wed Nov 30, 2016 2:19 pm

Just an additional note. There is a fallback implemented when then bandwidth-server is not reachable.
I am testing it with /tools netwatch and setting a global variable $"speedtest-server" to "up" or "down".

add comment=SPEEDTEST-SERVER down-script=":delay 15s\r\
    \n\r\
    \n#downscript\r\
    \n:global \"speedtest-server\"\r\
    \n\r\
    \n:if ([/ping 207.32.195.2 count=3] = 0) do={\r\
    \n\t:log error \"SPEEDTEST-SERVER is DOWN - skipping queue-calc script!\"\
    \r\
    \n\t:set \$\"speedtest-server\" \"down\"\r\
    \n}" host=207.32.195.2 up-script="#upscript\r\
    \n:global \"speedtest-server\"\r\
    \n\r\
    \n:if (\$\"speedtest-server\" = \"down\") do={\r\
    \n\t:log info \"SPEEDTEST-SERVER is UP\"\r\
    \n\t:set \$\"speedtest-server\" \"up\"\r\
    \n}"

That $"speedtest-server" variable is set by the netwatch tool and is used in my script at this stage:

#bandwidth determination
:global "speedtest-server"
:local txAvg
:local rxAvg
:local UPpercent 95
:local DOWNpercent 80
#checking if speedtest-server is up
:if ($"speedtest-server" != "down") do={
	#measuring tx rx bandwidth
	/tool bandwidth-test 207.32.195.2 duration=20s protocol=tcp direction=both user=btest password=btest do={
		:if ($txAvg < $"tx-10-second-average") do={
			:set txAvg $"tx-10-second-average"
		}
		:if ($rxAvg < $"rx-10-second-average") do={
			:set rxAvg $"rx-10-second-average"
		}
	}
} else={
#set static bandwidth values
:set txAvg 9500000
:set rxAvg 85500000
}

bajodel · Thu Dec 01, 2016 9:15 pm

just a note, imho you should measure total bandwidth on your real wan interface and not the test bandwidth avg in order to take the right script decisions

dash · Thu Dec 01, 2016 9:20 pm

I didn't get your point... Can you please explain a bit more in detail? Any valuable information is highly appreciated!

The Router has just one WAN Interface which is being used in the measurement. Before the measurement starts I reduce the bandwidth of all subqueues to a minimum.
After that the RB3011 itself is doing the measurement. There is no mangle rule applied to the traffic while the measurement is running. Im taking the 10s average because this gives me the most reliable results.

I know that it would cause issues if the bandwidth server is having problems and could not deliver enough data. But until today that wasnt the case... do you mean that?

measured WAN bandwidth + dynamic queue calculation script

measured WAN bandwidth + dynamic queue calculation script

Re: measured WAN bandwidth + dynamic queue calculation script

Re: measured WAN bandwidth + dynamic queue calculation script

Re: measured WAN bandwidth + dynamic queue calculation script

Who is online