QoS i2p priorities don't seem to work

RouterOS Beginner Basics
1

Hello. I have a problem with Queue Tree based QoS. Priorities set in queue tree don’t really seem to do the job. I’m running www server at home which is also i2p router (similar thing to Tor). Whereas I’d like i2p to use all “unused” bandwidth, allowing it to use anything above 70% of internet speed makes whole network unusable. Especially traffic incoming to www server. It even sometimes times out! It’s unacceptable because nobody can actually enter my website because i2p is hogging whole network. Internet in home is also highly inconvenient to use. Websites load like 6-8 seconds whereas without i2p traffic it’s much below 1 sec.

I’m using following rules in Mangle:

 0    ;;; ssl
      chain=prerouting action=mark-connection new-connection-mark=ssl 
      passthrough=yes protocol=tcp port=22,223,226
 1    chain=prerouting action=mark-packet new-packet-mark=ssl
      passthrough=yes connection-mark=ssl
 2    ;;; www
      chain=prerouting action=mark-connection new-connection-mark=www 
      passthrough=yes protocol=tcp port=80,806,803,443,4433,4436,8080,8100-8199
 3    chain=prerouting action=mark-connection new-connection-mark=www 
      passthrough=yes protocol=udp port=53 log=no log-prefix="" 
 4    chain=prerouting action=mark-packet new-packet-mark=www 
      passthrough=yes connection-mark=www log=no log-prefix="" 
 5    ;;; pub
      chain=prerouting action=mark-connection new-connection-mark=pub 
      passthrough=yes in-interface=wlan2-public log=no log-prefix="" 
 6    chain=prerouting action=mark-connection new-connection-mark=pub 
      passthrough=yes src-address=192.168.3.0/24
 7    chain=prerouting action=mark-connection new-connection-mark=pub 
      passthrough=yes dst-address=192.168.3.0/24
 8    chain=prerouting action=mark-packet new-packet-mark=pub 
      passthrough=yes connection-mark=pub log=no log-prefix="" 
 9    ;;; i2p
      chain=prerouting action=mark-connection new-connection-mark=i2p 
      passthrough=yes protocol=tcp port=23998,29733,29736
10    chain=prerouting action=mark-connection new-connection-mark=i2p 
      passthrough=yes protocol=udp port=23998,29733,29736
11    chain=prerouting action=mark-packet new-packet-mark=i2p-in 
      passthrough=yes dst-address=192.168.0.0/16 connection-mark=i2p
12    chain=prerouting action=mark-packet new-packet-mark=i2p-out 
      passthrough=yes dst-address=!192.168.0.0/16 connection-mark=i2p
13    chain=prerouting action=mark-packet new-packet-mark=nuc-out 
      passthrough=yes src-address=192.168.2.4 dst-address=!192.168.0.0/16 packet-mark=no-mark

It’s difficult to mangle i2p because it’s using random, non-ephemeral ports to communicate that’s why there’s “catch all” equivalent (#13). I have highly asymmetric internet (60/6) and i2p is more or less symmetric so it doesn’t really matter that there’s no incoming catch all rule as it’s limited by outgoing before reaching internet capacity.

And Queue tree:

 0   name="ssl" parent=global packet-mark=ssl limit-at=10M queue=default-small 
     priority=1 max-limit=4096M burst-limit=0 burst-threshold=0 burst-time=0s 
 1   name="spam" parent=global packet-mark=no-mark limit-at=512k queue=default-huge 
     priority=4 max-limit=1024M burst-limit=0 burst-threshold=0 burst-time=0s 
 2   name="pub" parent=global packet-mark=pub limit-at=512k queue=pcq-download-default 
     priority=3 max-limit=20M burst-limit=40M burst-threshold=16M burst-time=20s 
 3   name="low-out" parent=global packet-mark=i2p-out,nuc-out limit-at=0 queue=default-huge 
     priority=5 max-limit=3500k burst-limit=4M burst-threshold=3M burst-time=6s 
 4   name="www" parent=global packet-mark=www limit-at=1024k queue=default-huge 
     priority=2 max-limit=1024M burst-limit=0 burst-threshold=0 burst-time=0s 
 5   name="low-in" parent=global packet-mark=i2p-in limit-at=0 queue=default-huge 
     priority=5 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s

i2p has even lower priority than “spam” yet still it somehow hogs network. :confused:
I’d like to set i2p limit to 5M because that’s a bit below max upload of my internet but 4M already makes network barely usable, 3.5M is highly disturbing but at least friends don’t report www timeouts. However still they say it takes up to 20 seconds to load website whereas without i2p it’s less than 2 sec. It’s especially important to me as I’ll be applying for a job and there’s my portfolio on my server so if employer would try to enter my website and he’d receive timeout it’d be terrible failure. :frowning: People even stopped using my public wifi recently probably because it’s barely usable.

It makes me think that maybe i screw something up in setup? www and i2p router both are running on 192.168.2.4 server.

Side note - I know this QoS may seem funny because of “spam” being everything apart from ssl,i2p,www and public but actually I’m tunneling like 90% of traffic - VNC, NFS and basically all files transfer, Xpra, X server, pulseaudio, sometimes even http through ssh and as I can’t really differ ssh, all this traffic is just top priority.

2 
name="ssl" parent=global packet-mark=ssl limit-at=10M queue=default-small
     priority=1 max-limit=4096M burst-limit=0 burst-threshold=0 burst-time=0s

You’re setting Max-limit to 4Gbps there… also 1Gbps on spam and www queues. Guess you intended to be max-limit=4M and 1M respectively.

3

It’s ssl, super top priority. No i just wanted to set some limit-at and it requires max limit to be set to anything so i just set it to some hilarious value which won’t ever limit it. It’s because again those priorities don’t really seem to work so i tried various settings to make ssl as unlimited and “blocking” as it’s possible. I’m not sure if limit-at even should work for top priority queue, especially if it’s the only top priority queue so doesn’t have any competitor. I think not and those properties are pointless.

4

Queue Tree doesn’t work that way.

You have to set a Max-Limit on your queue tree parents (60 for download and 6 for upload), then make child queues whose Limit-at values would be enforced by priority order when parent queue reaches Max-Limit (saturation).

Child queues Max-Limit mean “use up to this limit if there’s spare bandwidth after all Limit-At child queues values are satisfied”, again, by queue priority.

Additionally, you have to set passthrough=no after connection marking rules (on packet marking rules just after), otherwise their marks can get overwritten down the way of traversing mangle rules if another rule matches.

5

ooooh… now it makes sense. Can I give multiple marks to single packet and combine them for queue? Because whereas some traffic has priorities valid only in external traffic, some (especially ssl) are rather related to LAN traffic. Or i need to mark internal and external ssl separately?

How “global” parent is handled then?

6

No, packets can have only one mark.

No need to mark upload/download traffic seperately, QoS will know which queue parent/child matches by its “top” parent (packet flow direction).

global its usually used as the parent on the top queue tree queue for download, so that no matter “who” or which interface is downloading, it’s taken into account globally, whereas the WAN interface is set as the parent for the upload parent queue as it’s the only way a packet flowing to “uplink” can travel.

7

So instead of marking incoming / outgoing packets i can just assume WAN interface parent will be upload and global download?
Uh… is it really okay? I mean then global will catch both upload and download? As well as internal traffic and basically all the traffic including LAN. Setting parent as WAN interface will only catch packets incoming FROM this interface? It sounds quite dirty i guess.

8

Nope… you can only control what leaves your router.

You mark once, then queue tree will only catch packets leaving the router through the parent.

global will take care of traffic globally leaving your router through any interface (download).

wan will take care of traffic leaving your router specifically through the WAN, which indirectly “globalizes” all upload traffic.

Have a look at http://mum.mikrotik.com/presentations/IT14/giordano.pdf

9

So i can’t shape incoming traffic? As in - if I have public wifi I can’t limit download bandwidth for it? Only upload? (or at least dropping his download won’t actually speed up my download, just slow down his and in fact as it’ll probably make him use bandwidth for longer period finally dropping his download would only make me wait longer till he stops hogging bandwidth?) So if lets say someone in public wifi would start downloading torrents I can’t prevent him from hogging my download?

That’s… a bit sad :frowning: Is there any trick to indirectly cap download for certain subnet?

For example the same situation takes place in case of ISP. Let’s say some consumer generates by his requests traffic incoming to ISP network which is much higher than his download capabilities. What happens to it? I mean like - server is sending 10gbps to client which has 10mbps download capabilities according to his data plan? There’s nothing you can do to prevent or at least suggest this server to stop flooding network?

10

Usually, limits over specific customers are done by simple queues, and shaping (QoS) by using Queue Tree.

You can limit per interface by using simple-queues (public wifi) or by setting an overall restriction if using hotspot, or limiting by hotspot client… all creates simple-queues.

Same goes for subnet (simple-queues).

Have a look at http://mum.mikrotik.com/presentations/US09/megis_qos.pdf (bear in mind this covered ROS 5, and things have significantly changed on 6)

You can’t really control what reaches your router, but restrict its travel through it, i.e. shape incoming (download) traffic.

Think about a doorman at a club, he cannot control the line, but the ratio/speed of clubbers accessing the club.

The approach I usually use is mark by traffic categories, asigning higher priorities depending on interactiveness of the protocol: e.g. 1st icmp, 2nd dns, 3d VoiP/Gaming, 4 HTTP, 5 http-download (by using connection-bytes to mark http downloads that have transfer more than X Mbs, like youtube, megaupload, etc), 6 P2P, and so on.

When doing QoS its crucial that you make sure all traffic gets into the queues, otherwise that will screw up all the QoS calculations.

That’s why when building the queues, I leave an additional one for “unmarked” traffic, with the lowest priority (8), a minimum Limit-at, and Max-limit as you wish.

This way, if there’s saturation, P2P and unmarked queues get just what you specified on their Limit-at values. If not, every queue will get its Max-limit value.

This makes everything less complex, makes everyone happy (hint: queue-type PCQ), and as you pointed out, minimizes “backlog” in established downloads.

11

Now I think it actually works. I’ve hogged 100% of bandwidth with ssh and my web server was still easily accessible, and ssh transfer was dropping to allow others hit limit-at:

 0   name="up" parent=global packet-mark="" limit-at=0 queue=default-huge priority=1 max-limit=5400k burst-limit=0 burst-threshold=0 burst-time=0s 
 1   name="down" parent=global packet-mark="" limit-at=0 queue=default-huge priority=2 max-limit=50M burst-limit=0 burst-threshold=0 burst-time=0s 
 2   name="down-2-www" parent=down packet-mark=down-www limit-at=1024k queue=default-huge priority=2 max-limit=1G burst-limit=0 burst-threshold=0 burst-time=0s 
 3   name="down-1-ssh" parent=down packet-mark=down-ssh limit-at=10M queue=default-huge priority=1 max-limit=1G burst-limit=0 burst-threshold=0 burst-time=0s 
 4   name="down-3-pub" parent=down packet-mark=down-pub limit-at=512k queue=pcq-download-default priority=3 max-limit=20M burst-limit=40M burst-threshold=16M burst-time=20s 
 5   name="down-4-p2p" parent=down packet-mark=down-p2p limit-at=5M queue=default-huge priority=4 max-limit=40M burst-limit=0 burst-threshold=0 burst-time=0s 
 6   name="down-5" parent=down packet-mark=down-i2p,down limit-at=1M queue=default-huge priority=5 max-limit=1G burst-limit=0 burst-threshold=0 burst-time=0s 
 7   name="up-2-www" parent=up packet-mark=up-www limit-at=512k queue=default-huge priority=2 max-limit=1G burst-limit=0 burst-threshold=0 burst-time=0s 
 8   name="up-1-ssh" parent=up packet-mark=up-ssh limit-at=1M queue=default-huge priority=1 max-limit=1G burst-limit=0 burst-threshold=0 burst-time=0s 
 9   name="up-3-pub" parent=up packet-mark=up-pub limit-at=128k queue=pcq-upload-default priority=3 max-limit=2M burst-limit=4M burst-threshold=1500k burst-time=20s 
10   name="up-4-p2p" parent=up packet-mark=up-p2p limit-at=128k queue=default-huge priority=4 max-limit=2M burst-limit=0 burst-threshold=0 burst-time=0s 
11   name="up-5" parent=up packet-mark=up-i2p,up limit-at=1M queue=default-huge priority=5 max-limit=4500k burst-limit=1G burst-threshold=4M burst-time=10s

Also rebuilt mangle: http://pastebin.com/Da5h7MWP

Is it correct now? I’m still mangling down / up using custom chains. I didn’t really get this interface parent trick. But well traditional mangle does the job i guess. I know www handling is quite simple here but still I think it’s a good start. Maybe I’ll make it more detailed as you suggested (www-download and dns separation) later

12

Once you mark a connection then its subsequent packets, ROS already know which queue tree top branch to place the traffic by looking at the interface it is going to exit; if for example you set for upload

name="up" parent=WAN_interface limit-at=0 queue=default-huge priority=1 max-limit=5400k burst-limit=0 burst-threshold=0 burst-time=0s

Packets marked with marks matched by “up” child queues whose out interface is “WAN_interface” will automatically “enter” this queue tree.

This simplifies mangling a lot, streamlines your configuration and optimizes CPU usage.