Strange Mangle situation - Download fighting Upload

RouterOS General
1

Hello there,

I have noticed a strange mangle situation that messes my whole Queue Tree, where downloads appear into uploads. To investigate this i disabled every single mangle rule except the 4 basic that split Donwload - Upload.

My setup is:
Hardware: RB962
Software: 6.44.3 ( stable )

WAN = Eth1 = PPPOE
LAN = Eth2, Eth3, Eth4, Eth5 and the 2 WIFI radios all Bridged

So my basic mangle rules are

/ip firewall mangle
add action=mark-connection chain=prerouting comment="All Download" connection-mark=no-mark in-interface=all-ppp new-connection-mark=Download passthrough=no
add action=mark-packet chain=prerouting connection-mark=Download new-packet-mark=Download passthrough=no
add action=mark-connection chain=postrouting comment="All Upload" connection-mark=no-mark new-connection-mark=Upload out-interface=all-ppp passthrough=no
add action=mark-packet chain=postrouting connection-mark=Upload new-packet-mark=Upload passthrough=no

As it is if i start for a test a HTTP download the traffic marked as Upload …
What am i missing here ?

2

Not at my pc at the moment, but below with you doing some reading on wiki should get you there.

You should not specify interfaces in mangle rules, then in queue tree config, specify the interface / queue as required, i.e. Bridge interface for download and PPPoE interface for upload

3

Thanks for reply but..

Either im doing something wrong or i hit into a mangle bug. I have read the wiki numerous times and try to read anything more i could find but here i am.
As for the specification of interfaces i think you are wrong and this isnt my problem. There is no way to specify interfaces in Queue trees as it seems..


.
queues.jpg
I will appreciate it if anyone else have any pointers or directions.

4

As a plus info, i also tried for the upload instead of marking connections and packets based on out interface to mark based on in interface which is “bridge”

add action=mark-connection chain=prerouting comment="All Upload" connection-mark=no-mark new-connection-mark=Upload in-interface=bridge passthrough=no
add action=mark-packet chain=prerouting connection-mark=Upload new-packet-mark=Upload passthrough=no

Again i tried to download a HTTP file and again ! the download marked as upload… I was pulling my hairs off .. Restarted the router without any other modification started the HTTP download again, this time is marked as correctly as Download.

Its a total mess maybe the router isnt taking the changes without reboot or there is a bug that marks the packets wrong randomly, i cant really say.. This have become a total jeopardy..

5

Hello,

As a debugging tool, I would suggest you check “log” and add meaningful log prefixes. You may get a hint as to which mangle rules are used (and when) and you’ll see which interfaces are used also.

To compare, I went about it the easy (and bad) way of only doing packet marking (no prior connection marking) and it works for me.

Sent from my cell phone. Sorry for the errors.

6

You have made a “global” queue instead of attaching it to an interface.
This sometimes has an application but usually it is not the best way.
Check the “parent” of the topmost item in each queue tree (Upload and Download in your example).

7

Yeap i tried what you said and i think i found a way to fix this. Or at least it didn’t messed my setup yet.

What i changed in my configuration and seems to identify and mark the flow right as download and upload is that instead of going “full bad” and mark straight only packets and no connections i just put the same interface info in packet marking as well.

For example instead of :

add action=mark-connection chain=prerouting comment=“All Upload” > in-interface=bridge > connection-mark=no-mark new-connection-mark=Upload passthrough=no
add action=mark-packet chain=prerouting connection-mark=Upload new-packet-mark=Upload passthrough=no

I did

add action=mark-connection chain=prerouting comment=“All Upload” > in-interface=bridge > new-connection-mark=Upload passthrough=yes
add action=mark-packet chain=prerouting connection-mark=Upload > in-interface=bridge > new-packet-mark=Upload passthrough=no

So long this did the trick, but i had the assumption that what ever you put into Connection marking follows the Packet marking if you use “Connection marking” as input ?
This had me fighting for a very long time and i hope it help others as well. Also this proves that almost every Tutorial out there is “Wrong” and the setups are working based on randomness and luck.

Maybe someone with the know-how can explain us why is this happening.

8

Oooor! Just dawned on me. Try checking passthrough for the connection marking rule. Can’t remember if the mark connection action allows for further processing.

Sent from my cell phone. Sorry for the errors.

9

Isn’t connection considered one as a whole regardless the direction of packet flow? Meaning that any connection initiated from LAN will be marked as upload because initial packet will be flowing from LAN towards WAN. Likewise connections initiated by some internet hosts will be marked as downlink.
The bulk amount of data transferred in each direction doesn’t define the direction of the connection.

10

For the connections i found that i have to have passthrough enabled and disabled for packet marking.

11

It is possible to use separate connection marks for upload and download based on incoming interface, but it is important that the matching rule includes the check for no-mark, so the mark is assigned only once (on the first match for the connection) and not overwritten later.
The above example fails to do that.

12

This plain simple doesn’t work at all. I tried it over and over again, if you check for “no mark” connection mark it captures almost nothing. In theory should work but it doesnt.
I really debating myself if what i read understood was right or wrong or actually RouterOS is doing its things …

Cant really tell…

13

Wow, seriously, all is wrong, including the Wiki?

Anyway, below an example, please note NO “Interfaces” in the Mangle rules (see mkx’s explanation in post above) and “Interfaces” in the Queue Tree config"

/ip firewall mangle
add action=mark-connection chain=prerouting connection-mark=no-mark connection-state=new new-connection-mark=video_con passthrough=yes protocol=tcp src-address-list=VideoMon
add action=mark-packet chain=forward connection-mark=video_con new-packet-mark=video_pkt passthrough=no

/queue tree
add max-limit=17M name=Up_Q parent=ether1 queue=default
add limit-at=2M max-limit=5M name=Video_Up packet-mark=video_pkt parent=Up_Q queue=Video_Qs
add max-limit=17M name=Down_Q parent=bridge1 queue=default
add limit-at=5M max-limit=10M name=Video_Down packet-mark=video_pkt parent=Down_Q queue=Video_Qs
14

You lost me there.

The subject here and my problem was to Categorize “ALL INTERNET DOWNLOAD” and “ALL INTERNET UPLOAD” as i had downloads marked as uploads and ofcource then everything else in the queue tree wasn’t working as it should.

**Your example cant work in my setup-situation because if i traffic shape based on interfaces i will slow down even local traffic as my router acting also as a bridge + switch. This is though another story and irrelevant to my problem.

*** Also what exactly is your mangle rule does ? Marking every new connection that isn’t marked before as video-con ? I dont get how this is going to work

15

Ok this way of marking separates incoming and outgoing connections, but that is not what you need.
You should stop marking connections as “upload” or “download”, that does not make sense. Mark connections to prioritize some over others.
To separate upload and download (as a traffic direction), use the queue tree parent.

16

Hy

I’m also confused. How can CZFan’s example work? The mangle uses src-address-list, meaning “Download” traffic from a bunch of IPs. How can those packet marks be used in Queue trees for uploads?

I found this which may help Chipburn.
https://mum.mikrotik.com/presentations/CZ09/QoS_Megis.pdf

My setup looks like this. May not be best practice as it looks up ALL traffic, but it works. I’ll clean it up one day… If the RB4011 starts choking :wink:

There might be some French comments so… pardon my French :smiley:

MANGLE

/ip firewall mangle
add action=passthrough chain=input comment="============Cote WAN QOS============" disabled=yes
add action=mark-packet chain=prerouting comment=L2TP-DOWN dst-port=1701,4500 in-interface=ether1 log-prefix=L2-D new-packet-mark=l2tp-down passthrough=no protocol=udp
add action=mark-packet chain=postrouting comment=L2TP-UP log-prefix=L2-U new-packet-mark=l2tp-up out-interface=ether1 passthrough=no protocol=udp src-port=1701,4500
add action=mark-packet chain=prerouting comment=Qnap-DOWN dst-address=172.16.2.254 in-interface=ether1 log-prefix=QD- new-packet-mark=qnap-down passthrough=no
add action=mark-packet chain=postrouting comment=Qnap-UP log-prefix=QU- new-packet-mark=qnap-up out-interface=ether1 passthrough=no src-address=172.16.2.254
add action=mark-packet chain=prerouting comment=General-DOWN in-interface=ether1 log-prefix=GD- new-packet-mark=general-down passthrough=no protocol=!ipsec-esp
add action=mark-packet chain=postrouting comment=General-UP log-prefix=GU- new-packet-mark=general-up out-interface=ether1 passthrough=no protocol=!ipsec-esp
add action=mark-packet chain=prerouting comment="===iPerf - VoIP-DOWN (l2tp) -- TEST PIPERF===" dst-address=172.16.3.254 in-interface=all-ppp log-prefix="vd===" new-packet-mark=\
    voip-down passthrough=no protocol=tcp src-address-list=00-PBX
add action=mark-packet chain=postrouting comment="===iPerf - VoIP-UP (l2tp) -- TEST PIPERF===" dst-address-list=00-PBX log-prefix="vu===" new-packet-mark=voip-up out-interface=\
    all-ppp passthrough=no protocol=tcp src-address=172.16.3.254
add action=mark-packet chain=prerouting comment="VoIP-DOWN (l2tp)" dst-address=172.16.1.0/24 in-interface=l2tp-Ste-Therese2 log-prefix="vd==" new-packet-mark=voip-down passthrough=\
    no protocol=udp src-address-list=00-PBX
add action=mark-packet chain=postrouting comment="VoIP-UP (l2tp)" dst-address-list=00-PBX log-prefix="vu==" new-packet-mark=voip-up out-interface=l2tp-Ste-Therese2 passthrough=no \
    protocol=udp src-address=172.16.1.0/24
add action=mark-packet chain=prerouting comment="GenL2TP-DOWN (l2tp)" in-interface=l2tp-Ste-Therese2 log-prefix=gl-down new-packet-mark=genl2tp-down passthrough=no protocol=\
    !ipsec-esp
add action=mark-packet chain=postrouting comment="GenL2TP-UP (l2tp)" log-prefix=gl-up new-packet-mark=genl2tp-up out-interface=l2tp-Ste-Therese2 passthrough=no protocol=!ipsec-esp

Queue Tree

/queue tree
add comment="TGV400 (50 upload)" limit-at=50M max-limit=50M name=TOP-UP parent=global
add limit-at=30M max-limit=50M name=General-U packet-mark=general-up parent=TOP-UP priority=3
add limit-at=10M max-limit=40M name=L2TP-U packet-mark=l2tp-up parent=TOP-UP priority=1
add comment="TGV400 (400 download)" limit-at=400M max-limit=400M name=TOP-DOWN parent=global
add limit-at=8M max-limit=38M name=General-l2tpU packet-mark=genl2tp-up parent=L2TP-U priority=3
add limit-at=285M max-limit=400M name=General-D packet-mark=general-down parent=TOP-DOWN priority=3
add limit-at=15M max-limit=30M name=L2TP-D packet-mark=l2tp-down parent=TOP-DOWN priority=1
add limit-at=13M max-limit=28M name=General-l2tpD packet-mark=genl2tp-down parent=L2TP-D priority=3
add limit-at=2M max-limit=2M name=VoIP-D packet-mark=voip-down parent=L2TP-D priority=1
add limit-at=2M max-limit=2M name=VoIP-U packet-mark=voip-up parent=L2TP-U priority=1
add comment="===LIMITATION DES INVITES===" name=G-DOWNLOAD packet-mark="M=DOWNLOAD" parent=global queue="Q=DOWNLOAD"
add name=G-UPLOAD packet-mark="M=UPLOAD" parent=global queue="Q=UPLOAD"
add limit-at=10M max-limit=40M name=Qnap-U packet-mark=qnap-up parent=TOP-UP priority=2
add limit-at=100M max-limit=200M name=Qnap-D packet-mark=qnap-down parent=TOP-DOWN priority=2
17

the mangle uses src-address-list, for the device starting the connection, in this case it was a pc that was used for monitoring security cctv cameras.
This connection and packets are marked, for both up and down traffic.
Then in queue tree config is where you apply the shaping/limits for relevant directions, i.e. The leaving interface will determine the direction, bridge for download traffic and ether1 (WAN) for upload traffic

18

I have solved my problem and marked my answer as solved, so we are a bit out of subject but out of curiosity :

How in the world are you going to specify flow direction in the “Queue Tree” ? The Flow Direction is done in mangle with packer marking and then used as an input in the “Queue Tree”.
I think that all this time you are speaking about “Simple Queues”… where you can input interfaces but this is a non efficient way of Queuing as every packet is checked in every Queue rule which can soon become a CPU hog.

19

I already told you that above: you should set the parent of the topmost items in the queue tree to the interfaces where you want the queue to operate.
The queue tree only operates in the sending direction. So you attach the Upload queue tree to your internet interface, and the Download queue tree to your LAN interface.
Now you have set them to “global” and then there is no concept of direction unless it is present in the packet marks.

20

QtreeIface.JPG