Community discussions

 
Chipburn
just joined
Topic Author
Posts: 17
Joined: Mon Nov 12, 2018 7:20 pm

Strange Mangle situation - Download fighting Upload

Thu Jun 06, 2019 12:49 pm

Hello there,

I have noticed a strange mangle situation that messes my whole Queue Tree, where downloads appear into uploads. To investigate this i disabled every single mangle rule except the 4 basic that split Donwload - Upload.

My setup is:
Hardware: RB962
Software: 6.44.3 ( stable )

WAN = Eth1 = PPPOE
LAN = Eth2, Eth3, Eth4, Eth5 and the 2 WIFI radios all Bridged

So my basic mangle rules are
/ip firewall mangle
add action=mark-connection chain=prerouting comment="All Download" connection-mark=no-mark in-interface=all-ppp new-connection-mark=Download passthrough=no
add action=mark-packet chain=prerouting connection-mark=Download new-packet-mark=Download passthrough=no
add action=mark-connection chain=postrouting comment="All Upload" connection-mark=no-mark new-connection-mark=Upload out-interface=all-ppp passthrough=no
add action=mark-packet chain=postrouting connection-mark=Upload new-packet-mark=Upload passthrough=no
As it is if i start for a test a HTTP download the traffic marked as Upload ....
What am i missing here ?
 
User avatar
CZFan
Forum Guru
Forum Guru
Posts: 1229
Joined: Sun Oct 09, 2016 8:25 pm
Location: South Africa, Randburg
Contact:

Re: Strange Mangle situation - Download fighting Upload

Fri Jun 07, 2019 12:12 am

Not at my pc at the moment, but below with you doing some reading on wiki should get you there.

You should not specify interfaces in mangle rules, then in queue tree config, specify the interface / queue as required, i.e. Bridge interface for download and PPPoE interface for upload
MTCNA, MTCTCE, MTCRE & MTCINE
 
Chipburn
just joined
Topic Author
Posts: 17
Joined: Mon Nov 12, 2018 7:20 pm

Re: Strange Mangle situation - Download fighting Upload

Fri Jun 07, 2019 4:18 pm

Not at my pc at the moment, but below with you doing some reading on wiki should get you there.

You should not specify interfaces in mangle rules, then in queue tree config, specify the interface / queue as required, i.e. Bridge interface for download and PPPoE interface for upload
Thanks for reply but..

Either im doing something wrong or i hit into a mangle bug. I have read the wiki numerous times and try to read anything more i could find but here i am.
As for the specification of interfaces i think you are wrong and this isnt my problem. There is no way to specify interfaces in Queue trees as it seems..


.
queues.jpg

I will appreciate it if anyone else have any pointers or directions.
You do not have the required permissions to view the files attached to this post.
 
Chipburn
just joined
Topic Author
Posts: 17
Joined: Mon Nov 12, 2018 7:20 pm

Re: Strange Mangle situation - Download fighting Upload

Fri Jun 07, 2019 5:13 pm

As a plus info, i also tried for the upload instead of marking connections and packets based on out interface to mark based on in interface which is "bridge"
add action=mark-connection chain=prerouting comment="All Upload" connection-mark=no-mark new-connection-mark=Upload in-interface=bridge passthrough=no
add action=mark-packet chain=prerouting connection-mark=Upload new-packet-mark=Upload passthrough=no
Again i tried to download a HTTP file and again ! the download marked as upload..... I was pulling my hairs off .. Restarted the router without any other modification started the HTTP download again, this time is marked as correctly as Download.

Its a total mess maybe the router isnt taking the changes without reboot or there is a bug that marks the packets wrong randomly, i cant really say.. This have become a total jeopardy..
 
User avatar
AlainCasault
Trainer
Trainer
Posts: 586
Joined: Fri Apr 30, 2010 3:25 pm
Location: Laval, QC, Canada
Contact:

Re: Strange Mangle situation - Download fighting Upload

Fri Jun 07, 2019 5:32 pm

Hello,

As a debugging tool, I would suggest you check "log" and add meaningful log prefixes. You may get a hint as to which mangle rules are used (and when) and you'll see which interfaces are used also.

To compare, I went about it the easy (and bad) way of only doing packet marking (no prior connection marking) and it works for me.

Sent from my cell phone. Sorry for the errors.

___________________________
Alain Casault, Eng.
If I helped you, let me know!
 
pe1chl
Forum Guru
Forum Guru
Posts: 5276
Joined: Mon Jun 08, 2015 12:09 pm

Re: Strange Mangle situation - Download fighting Upload

Fri Jun 07, 2019 5:36 pm

Either im doing something wrong or i hit into a mangle bug. I have read the wiki numerous times and try to read anything more i could find but here i am.
You have made a "global" queue instead of attaching it to an interface.
This sometimes has an application but usually it is not the best way.
Check the "parent" of the topmost item in each queue tree (Upload and Download in your example).
 
Chipburn
just joined
Topic Author
Posts: 17
Joined: Mon Nov 12, 2018 7:20 pm

Re: Strange Mangle situation - Download fighting Upload  [SOLVED]

Fri Jun 07, 2019 5:57 pm

Hello,

As a debugging tool, I would suggest you check "log" and add meaningful log prefixes. You may get a hint as to which mangle rules are used (and when) and you'll see which interfaces are used also.

To compare, I went about it the easy (and bad) way of only doing packet marking (no prior connection marking) and it works for me.

Sent from my cell phone. Sorry for the errors.

Yeap i tried what you said and i think i found a way to fix this. Or at least it didn't messed my setup yet.

What i changed in my configuration and seems to identify and mark the flow right as download and upload is that instead of going "full bad" and mark straight only packets and no connections i just put the same interface info in packet marking as well.

For example instead of :
add action=mark-connection chain=prerouting comment="All Upload" in-interface=bridge connection-mark=no-mark new-connection-mark=Upload passthrough=no
add action=mark-packet chain=prerouting connection-mark=Upload new-packet-mark=Upload passthrough=no
I did
add action=mark-connection chain=prerouting comment="All Upload" in-interface=bridge new-connection-mark=Upload passthrough=yes
add action=mark-packet chain=prerouting connection-mark=Upload in-interface=bridge new-packet-mark=Upload passthrough=no

So long this did the trick, but i had the assumption that what ever you put into Connection marking follows the Packet marking if you use "Connection marking" as input ?
This had me fighting for a very long time and i hope it help others as well. Also this proves that almost every Tutorial out there is "Wrong" and the setups are working based on randomness and luck.

Maybe someone with the know-how can explain us why is this happening.
 
User avatar
AlainCasault
Trainer
Trainer
Posts: 586
Joined: Fri Apr 30, 2010 3:25 pm
Location: Laval, QC, Canada
Contact:

Re: Strange Mangle situation - Download fighting Upload

Fri Jun 07, 2019 6:39 pm

Oooor! Just dawned on me. Try checking passthrough for the connection marking rule. Can't remember if the mark connection action allows for further processing.

Sent from my cell phone. Sorry for the errors.

___________________________
Alain Casault, Eng.
If I helped you, let me know!
 
mkx
Forum Guru
Forum Guru
Posts: 2278
Joined: Thu Mar 03, 2016 10:23 pm

Re: Strange Mangle situation - Download fighting Upload

Fri Jun 07, 2019 6:46 pm

Isn't connection considered one as a whole regardless the direction of packet flow? Meaning that any connection initiated from LAN will be marked as upload because initial packet will be flowing from LAN towards WAN. Likewise connections initiated by some internet hosts will be marked as downlink.
The bulk amount of data transferred in each direction doesn't define the direction of the connection.
BR,
Metod
 
Chipburn
just joined
Topic Author
Posts: 17
Joined: Mon Nov 12, 2018 7:20 pm

Re: Strange Mangle situation - Download fighting Upload

Fri Jun 07, 2019 7:20 pm

Oooor! Just dawned on me. Try checking passthrough for the connection marking rule. Can't remember if the mark connection action allows for further processing.

Sent from my cell phone. Sorry for the errors.
For the connections i found that i have to have passthrough enabled and disabled for packet marking.
 
pe1chl
Forum Guru
Forum Guru
Posts: 5276
Joined: Mon Jun 08, 2015 12:09 pm

Re: Strange Mangle situation - Download fighting Upload

Fri Jun 07, 2019 7:21 pm

It is possible to use separate connection marks for upload and download based on incoming interface, but it is important that the matching rule includes the check for no-mark, so the mark is assigned only once (on the first match for the connection) and not overwritten later.
The above example fails to do that.
 
Chipburn
just joined
Topic Author
Posts: 17
Joined: Mon Nov 12, 2018 7:20 pm

Re: Strange Mangle situation - Download fighting Upload

Fri Jun 07, 2019 7:43 pm

It is possible to use separate connection marks for upload and download based on incoming interface, but it is important that the matching rule includes the check for no-mark, so the mark is assigned only once (on the first match for the connection) and not overwritten later.
The above example fails to do that.
This plain simple doesn't work at all. I tried it over and over again, if you check for "no mark" connection mark it captures almost nothing. In theory should work but it doesnt.
I really debating myself if what i read understood was right or wrong or actually RouterOS is doing its things ....

Cant really tell...
 
User avatar
CZFan
Forum Guru
Forum Guru
Posts: 1229
Joined: Sun Oct 09, 2016 8:25 pm
Location: South Africa, Randburg
Contact:

Re: Strange Mangle situation - Download fighting Upload

Fri Jun 07, 2019 7:52 pm

...
So long this did the trick, but i had the assumption that what ever you put into Connection marking follows the Packet marking if you use "Connection marking" as input ?
This had me fighting for a very long time and i hope it help others as well. Also this proves that almost every Tutorial out there is "Wrong" and the setups are working based on randomness and luck.

Maybe someone with the know-how can explain us why is this happening.
Wow, seriously, all is wrong, including the Wiki?

Anyway, below an example, please note NO "Interfaces" in the Mangle rules (see mkx's explanation in post above) and "Interfaces" in the Queue Tree config"

/ip firewall mangle
add action=mark-connection chain=prerouting connection-mark=no-mark connection-state=new new-connection-mark=video_con passthrough=yes protocol=tcp src-address-list=VideoMon
add action=mark-packet chain=forward connection-mark=video_con new-packet-mark=video_pkt passthrough=no

/queue tree
add max-limit=17M name=Up_Q parent=ether1 queue=default
add limit-at=2M max-limit=5M name=Video_Up packet-mark=video_pkt parent=Up_Q queue=Video_Qs
add max-limit=17M name=Down_Q parent=bridge1 queue=default
add limit-at=5M max-limit=10M name=Video_Down packet-mark=video_pkt parent=Down_Q queue=Video_Qs

MTCNA, MTCTCE, MTCRE & MTCINE
 
Chipburn
just joined
Topic Author
Posts: 17
Joined: Mon Nov 12, 2018 7:20 pm

Re: Strange Mangle situation - Download fighting Upload

Fri Jun 07, 2019 8:02 pm

...
So long this did the trick, but i had the assumption that what ever you put into Connection marking follows the Packet marking if you use "Connection marking" as input ?
This had me fighting for a very long time and i hope it help others as well. Also this proves that almost every Tutorial out there is "Wrong" and the setups are working based on randomness and luck.

Maybe someone with the know-how can explain us why is this happening.
Wow, seriously, all is wrong, including the Wiki?

Anyway, below an example, please note NO "Interfaces" in the Mangle rules (see mkx's explanation in post above) and "Interfaces" in the Queue Tree config"

/ip firewall mangle
add action=mark-connection chain=prerouting connection-mark=no-mark connection-state=new new-connection-mark=video_con passthrough=yes protocol=tcp src-address-list=VideoMon
add action=mark-packet chain=forward connection-mark=video_con new-packet-mark=video_pkt passthrough=no

/queue tree
add max-limit=17M name=Up_Q parent=ether1 queue=default
add limit-at=2M max-limit=5M name=Video_Up packet-mark=video_pkt parent=Up_Q queue=Video_Qs
add max-limit=17M name=Down_Q parent=bridge1 queue=default
add limit-at=5M max-limit=10M name=Video_Down packet-mark=video_pkt parent=Down_Q queue=Video_Qs

You lost me there.

The subject here and my problem was to Categorize "ALL INTERNET DOWNLOAD" and "ALL INTERNET UPLOAD" as i had downloads marked as uploads and ofcource then everything else in the queue tree wasn't working as it should.

**Your example cant work in my setup-situation because if i traffic shape based on interfaces i will slow down even local traffic as my router acting also as a bridge + switch. This is though another story and irrelevant to my problem.

*** Also what exactly is your mangle rule does ? Marking every new connection that isn't marked before as video-con ? I dont get how this is going to work
Last edited by Chipburn on Fri Jun 07, 2019 8:58 pm, edited 2 times in total.
 
pe1chl
Forum Guru
Forum Guru
Posts: 5276
Joined: Mon Jun 08, 2015 12:09 pm

Re: Strange Mangle situation - Download fighting Upload

Fri Jun 07, 2019 8:58 pm

It is possible to use separate connection marks for upload and download based on incoming interface, but it is important that the matching rule includes the check for no-mark, so the mark is assigned only once (on the first match for the connection) and not overwritten later.
The above example fails to do that.
This plain simple doesn't work at all. I tried it over and over again, if you check for "no mark" connection mark it captures almost nothing. In theory should work but it doesnt.
Ok this way of marking separates incoming and outgoing connections, but that is not what you need.
You should stop marking connections as "upload" or "download", that does not make sense. Mark connections to prioritize some over others.
To separate upload and download (as a traffic direction), use the queue tree parent.
 
User avatar
AlainCasault
Trainer
Trainer
Posts: 586
Joined: Fri Apr 30, 2010 3:25 pm
Location: Laval, QC, Canada
Contact:

Re: Strange Mangle situation - Download fighting Upload

Sat Jun 08, 2019 12:45 am

Hy

I'm also confused. How can CZFan's example work? The mangle uses src-address-list, meaning "Download" traffic from a bunch of IPs. How can those packet marks be used in Queue trees for uploads?

I found this which may help Chipburn.
https://mum.mikrotik.com/presentations/ ... _Megis.pdf

My setup looks like this. May not be best practice as it looks up ALL traffic, but it works. I'll clean it up one day... If the RB4011 starts choking ;)

There might be some French comments so... pardon my French :D
___________________________________________________________________

MANGLE
/ip firewall mangle
add action=passthrough chain=input comment="============Cote WAN QOS============" disabled=yes
add action=mark-packet chain=prerouting comment=L2TP-DOWN dst-port=1701,4500 in-interface=ether1 log-prefix=L2-D new-packet-mark=l2tp-down passthrough=no protocol=udp
add action=mark-packet chain=postrouting comment=L2TP-UP log-prefix=L2-U new-packet-mark=l2tp-up out-interface=ether1 passthrough=no protocol=udp src-port=1701,4500
add action=mark-packet chain=prerouting comment=Qnap-DOWN dst-address=172.16.2.254 in-interface=ether1 log-prefix=QD- new-packet-mark=qnap-down passthrough=no
add action=mark-packet chain=postrouting comment=Qnap-UP log-prefix=QU- new-packet-mark=qnap-up out-interface=ether1 passthrough=no src-address=172.16.2.254
add action=mark-packet chain=prerouting comment=General-DOWN in-interface=ether1 log-prefix=GD- new-packet-mark=general-down passthrough=no protocol=!ipsec-esp
add action=mark-packet chain=postrouting comment=General-UP log-prefix=GU- new-packet-mark=general-up out-interface=ether1 passthrough=no protocol=!ipsec-esp
add action=mark-packet chain=prerouting comment="===iPerf - VoIP-DOWN (l2tp) -- TEST PIPERF===" dst-address=172.16.3.254 in-interface=all-ppp log-prefix="vd===" new-packet-mark=\
    voip-down passthrough=no protocol=tcp src-address-list=00-PBX
add action=mark-packet chain=postrouting comment="===iPerf - VoIP-UP (l2tp) -- TEST PIPERF===" dst-address-list=00-PBX log-prefix="vu===" new-packet-mark=voip-up out-interface=\
    all-ppp passthrough=no protocol=tcp src-address=172.16.3.254
add action=mark-packet chain=prerouting comment="VoIP-DOWN (l2tp)" dst-address=172.16.1.0/24 in-interface=l2tp-Ste-Therese2 log-prefix="vd==" new-packet-mark=voip-down passthrough=\
    no protocol=udp src-address-list=00-PBX
add action=mark-packet chain=postrouting comment="VoIP-UP (l2tp)" dst-address-list=00-PBX log-prefix="vu==" new-packet-mark=voip-up out-interface=l2tp-Ste-Therese2 passthrough=no \
    protocol=udp src-address=172.16.1.0/24
add action=mark-packet chain=prerouting comment="GenL2TP-DOWN (l2tp)" in-interface=l2tp-Ste-Therese2 log-prefix=gl-down new-packet-mark=genl2tp-down passthrough=no protocol=\
    !ipsec-esp
add action=mark-packet chain=postrouting comment="GenL2TP-UP (l2tp)" log-prefix=gl-up new-packet-mark=genl2tp-up out-interface=l2tp-Ste-Therese2 passthrough=no protocol=!ipsec-esp

Queue Tree
/queue tree
add comment="TGV400 (50 upload)" limit-at=50M max-limit=50M name=TOP-UP parent=global
add limit-at=30M max-limit=50M name=General-U packet-mark=general-up parent=TOP-UP priority=3
add limit-at=10M max-limit=40M name=L2TP-U packet-mark=l2tp-up parent=TOP-UP priority=1
add comment="TGV400 (400 download)" limit-at=400M max-limit=400M name=TOP-DOWN parent=global
add limit-at=8M max-limit=38M name=General-l2tpU packet-mark=genl2tp-up parent=L2TP-U priority=3
add limit-at=285M max-limit=400M name=General-D packet-mark=general-down parent=TOP-DOWN priority=3
add limit-at=15M max-limit=30M name=L2TP-D packet-mark=l2tp-down parent=TOP-DOWN priority=1
add limit-at=13M max-limit=28M name=General-l2tpD packet-mark=genl2tp-down parent=L2TP-D priority=3
add limit-at=2M max-limit=2M name=VoIP-D packet-mark=voip-down parent=L2TP-D priority=1
add limit-at=2M max-limit=2M name=VoIP-U packet-mark=voip-up parent=L2TP-U priority=1
add comment="===LIMITATION DES INVITES===" name=G-DOWNLOAD packet-mark="M=DOWNLOAD" parent=global queue="Q=DOWNLOAD"
add name=G-UPLOAD packet-mark="M=UPLOAD" parent=global queue="Q=UPLOAD"
add limit-at=10M max-limit=40M name=Qnap-U packet-mark=qnap-up parent=TOP-UP priority=2
add limit-at=100M max-limit=200M name=Qnap-D packet-mark=qnap-down parent=TOP-DOWN priority=2
___________________________
Alain Casault, Eng.
If I helped you, let me know!
 
User avatar
CZFan
Forum Guru
Forum Guru
Posts: 1229
Joined: Sun Oct 09, 2016 8:25 pm
Location: South Africa, Randburg
Contact:

Re: Strange Mangle situation - Download fighting Upload

Sat Jun 08, 2019 2:25 am

Hy

I'm also confused. How can CZFan's example work? The mangle uses src-address-list, meaning "Download" traffic from a bunch of IPs. How can those packet marks be used in Queue trees for uploads?
...
the mangle uses src-address-list, for the device starting the connection, in this case it was a pc that was used for monitoring security cctv cameras.
This connection and packets are marked, for both up and down traffic.
Then in queue tree config is where you apply the shaping/limits for relevant directions, i.e. The leaving interface will determine the direction, bridge for download traffic and ether1 (WAN) for upload traffic
MTCNA, MTCTCE, MTCRE & MTCINE
 
Chipburn
just joined
Topic Author
Posts: 17
Joined: Mon Nov 12, 2018 7:20 pm

Re: Strange Mangle situation - Download fighting Upload

Sat Jun 08, 2019 11:12 am

Hy

I'm also confused. How can CZFan's example work? The mangle uses src-address-list, meaning "Download" traffic from a bunch of IPs. How can those packet marks be used in Queue trees for uploads?
...
the mangle uses src-address-list, for the device starting the connection, in this case it was a pc that was used for monitoring security cctv cameras.
This connection and packets are marked, for both up and down traffic.
Then in queue tree config is where you apply the shaping/limits for relevant directions, i.e. The leaving interface will determine the direction, bridge for download traffic and ether1 (WAN) for upload traffic
I have solved my problem and marked my answer as solved, so we are a bit out of subject but out of curiosity :

How in the world are you going to specify flow direction in the "Queue Tree" ? The Flow Direction is done in mangle with packer marking and then used as an input in the "Queue Tree".
I think that all this time you are speaking about "Simple Queues"...... where you can input interfaces but this is a non efficient way of Queuing as every packet is checked in every Queue rule which can soon become a CPU hog.
 
pe1chl
Forum Guru
Forum Guru
Posts: 5276
Joined: Mon Jun 08, 2015 12:09 pm

Re: Strange Mangle situation - Download fighting Upload

Sat Jun 08, 2019 12:17 pm

I already told you that above: you should set the parent of the topmost items in the queue tree to the interfaces where you want the queue to operate.
The queue tree only operates in the sending direction. So you attach the Upload queue tree to your internet interface, and the Download queue tree to your LAN interface.
Now you have set them to "global" and then there is no concept of direction unless it is present in the packet marks.
 
User avatar
CZFan
Forum Guru
Forum Guru
Posts: 1229
Joined: Sun Oct 09, 2016 8:25 pm
Location: South Africa, Randburg
Contact:

Re: Strange Mangle situation - Download fighting Upload

Sat Jun 08, 2019 7:21 pm

...

How in the world are you going to specify flow direction in the "Queue Tree" ? The Flow Direction is done in mangle with packer marking and then used as an input in the "Queue Tree".
...
QtreeIface.JPG
You do not have the required permissions to view the files attached to this post.
MTCNA, MTCTCE, MTCRE & MTCINE
 
sindy
Forum Guru
Forum Guru
Posts: 3287
Joined: Mon Dec 04, 2017 9:19 pm

Re: Strange Mangle situation - Download fighting Upload

Sat Jun 08, 2019 9:10 pm

I had the assumption that what ever you put into Connection marking follows the Packet marking if you use "Connection marking" as input ?
This had me fighting for a very long time and i hope it help others as well. Also this proves that almost every Tutorial out there is "Wrong" and the setups are working based on randomness and luck.

Maybe someone with the know-how can explain us why is this happening.
Slow down, man. Step by step.

A connection-mark is assigned to the whole connection, i.e. a (bi-directional by nature) TCP session or a UDP stream (which is usually also bi-directional). So once you assign connection-mark=XXX to a connection, by means of a mangle rule with action=mark-connection, all further packets belonging to that connection, regardless their direction, and including the packet which caused the connection-mark to be assigned if passthrough was set to yes in the action=mark-connection rule, match the condition connection-mark=XXX.

A connection-mark never "follows" a packet-mark automatically, nor vice versa - you have to use a mangle rule to generate one from the other, i.e. action=mark-packet connection-mark=XXX new-packet-mark=YYY (used more frequently as it makes sense in more scenarios) or action=mark-connection packet-mark=YYY new-connection-mark=XXX (which might make sense in some complex scenarios where the packet-mark is assigned by an /interface bridge filter rule). The name spaces of connection-mark, packet-mark, and routing-mark are independent, so use of the same string as connection-mark and packet-mark has no effect unless you use one of the rules above.

Next, don't misunderstand what @pe1chl has written - in the current version of RouterOS (6.44.3 for the record), only a single connection-mark can be assigned to a connection at a time. If you assign another connection-mark to it, the previously assigned one is overwritten. I don't exactly understand what @pe1chl had in mind when saying that you can assign a distinct connection-mark to each direction of a connection; what you can do is to assign a different connection-mark depending on the direction in which the connection was initiated, but that's not the same thing.

Another important point is that assignment of packet-mark and routing-mark is only valid for a single packet; the next packet belonging to the same direction of the same connection has to be packet-marked or routing-marked on its own if that is required (which it usually is).

So to assign the appropriate packet-mark to a packet, to be used as a key to select a queue, you have to use a mangle rule (usually, one assigning a packet-mark depending on connection-mark as above; to do so already for the first packet of a connection, the action=mark-connection rule must have passthrough set to yes and the action=mark-packet rule must follow it (and it doesn't necessarily need to have passthrough set to no as you may want to assign both a packet-mark and a routing-mark).

But you have two possibilities how to use the packet-mark to choose a queue from the tree:
  • either you set the parent of the queue to global, and in that case, you have to use a distinct packet-mark for each direction to choose the right queue, so you need to match something more than just the connection-mark in the mangle rule,
  • or you set the parent of the queue to an outgoing interface, and in that case, you can use the same packet-mark for both directions as the queue matching the packet-mark is only chosen among those whose ultimate parent matches the packet's output interface.
Instead of writing novels, post /export hide-sensitive. Use find&replace in your favourite text editor to systematically replace all occurrences of each public IP address potentially identifying you by a distinctive pattern such as my.public.ip.1.
 
User avatar
AlainCasault
Trainer
Trainer
Posts: 586
Joined: Fri Apr 30, 2010 3:25 pm
Location: Laval, QC, Canada
Contact:

Re: Strange Mangle situation - Download fighting Upload

Sat Jun 08, 2019 11:11 pm

Hy

I'm also confused. How can CZFan's example work? The mangle uses src-address-list, meaning "Download" traffic from a bunch of IPs. How can those packet marks be used in Queue trees for uploads?
...
the mangle uses src-address-list, for the device starting the connection, in this case it was a pc that was used for monitoring security cctv cameras.
This connection and packets are marked, for both up and down traffic.
Then in queue tree config is where you apply the shaping/limits for relevant directions, i.e. The leaving interface will determine the direction, bridge for download traffic and ether1 (WAN) for upload traffic

Gotchaaa!!! Connection, of course !!!! :D
___________________________
Alain Casault, Eng.
If I helped you, let me know!
 
Chipburn
just joined
Topic Author
Posts: 17
Joined: Mon Nov 12, 2018 7:20 pm

Re: Strange Mangle situation - Download fighting Upload

Sun Jun 09, 2019 8:19 pm

I had the assumption that what ever you put into Connection marking follows the Packet marking if you use "Connection marking" as input ?
This had me fighting for a very long time and i hope it help others as well. Also this proves that almost every Tutorial out there is "Wrong" and the setups are working based on randomness and luck.

Maybe someone with the know-how can explain us why is this happening.
Slow down, man. Step by step.

A connection-mark is assigned to the whole connection, i.e. a (bi-directional by nature) TCP session or a UDP stream (which is usually also bi-directional). So once you assign connection-mark=XXX to a connection, by means of a mangle rule with action=mark-connection, all further packets belonging to that connection, regardless their direction, and including the packet which caused the connection-mark to be assigned if passthrough was set to yes in the action=mark-connection rule, match the condition connection-mark=XXX.

A connection-mark never "follows" a packet-mark automatically, nor vice versa - you have to use a mangle rule to generate one from the other, i.e. action=mark-packet connection-mark=XXX new-packet-mark=YYY (used more frequently as it makes sense in more scenarios) or action=mark-connection packet-mark=YYY new-connection-mark=XXX (which might make sense in some complex scenarios where the packet-mark is assigned by an /interface bridge filter rule). The name spaces of connection-mark, packet-mark, and routing-mark are independent, so use of the same string as connection-mark and packet-mark has no effect unless you use one of the rules above.

Next, don't misunderstand what @pe1chl has written - in the current version of RouterOS (6.44.3 for the record), only a single connection-mark can be assigned to a connection at a time. If you assign another connection-mark to it, the previously assigned one is overwritten. I don't exactly understand what @pe1chl had in mind when saying that you can assign a distinct connection-mark to each direction of a connection; what you can do is to assign a different connection-mark depending on the direction in which the connection was initiated, but that's not the same thing.

Another important point is that assignment of packet-mark and routing-mark is only valid for a single packet; the next packet belonging to the same direction of the same connection has to be packet-marked or routing-marked on its own if that is required (which it usually is).

So to assign the appropriate packet-mark to a packet, to be used as a key to select a queue, you have to use a mangle rule (usually, one assigning a packet-mark depending on connection-mark as above; to do so already for the first packet of a connection, the action=mark-connection rule must have passthrough set to yes and the action=mark-packet rule must follow it (and it doesn't necessarily need to have passthrough set to no as you may want to assign both a packet-mark and a routing-mark).

But you have two possibilities how to use the packet-mark to choose a queue from the tree:
  • either you set the parent of the queue to global, and in that case, you have to use a distinct packet-mark for each direction to choose the right queue, so you need to match something more than just the connection-mark in the mangle rule,
  • or you set the parent of the queue to an outgoing interface, and in that case, you can use the same packet-mark for both directions as the queue matching the packet-mark is only chosen among those whose ultimate parent matches the packet's output interface.
Nice informations here, thank you, i didnt knew about the interface trick in the Queue Tree and it makes sense now that you and CZFan explained it.

Kind regards

Who is online

Users browsing this forum: No registered users and 44 guests