shaping + priorities - mangle and queue tree

[xordi]

Hello World Thes is my 1’st post. But it is about very important think.
No im testing my configuration on my home comp. I have on ether3 my ISP - named “WAN”
All other iface are in brige - “LAN”

After read tons of materials,on each kind forums, wiki’s page… i did something like that and i have some problems with that:

MANGLE(i think here is a problem):

0 ;;; WWW
chain=prerouting action=mark-connection new-connection-mark=all_conn_www passthrough=yes protocol=tcp src-port=80,443
1 ;;; www up
chain=prerouting action=mark-packet new-packet-mark=www_up passthrough=no in-interface=LAN connection-mark=all_conn_www
2 ;;; www down
chain=prerouting action=mark-packet new-packet-mark=www_down passthrough=no in-interface=WAN connection-mark=all_conn_www
3 ;;; ICMP
chain=prerouting action=mark-connection new-connection-mark=all_conn_icmp passthrough=yes protocol=icmp
4 ;;; icmp_up
chain=prerouting action=mark-packet new-packet-mark=icmp_up passthrough=no in-interface=LAN connection-mark=all_conn_icmp
5 ;;; icmp_down
chain=prerouting action=mark-packet new-packet-mark=icmp_down passthrough=no in-interface=WAN connection-mark=all_conn_icmp
6 ;;; P2P
chain=prerouting action=mark-connection new-connection-mark=all_conn_p2p passthrough=yes p2p=all-p2p
7 ;;; p2p up
chain=prerouting action=mark-packet new-packet-mark=p2p_up passthrough=no in-interface=LAN connection-mark=all_conn_p2p
8 ;;; p2p down
chain=prerouting action=mark-packet new-packet-mark=p2p_down passthrough=no in-interface=WAN connection-mark=all_conn_p2p
9 ;;; other
chain=prerouting action=mark-connection new-connection-mark=all_conn_other passthrough=yes
10 ;;; other_up
chain=prerouting action=mark-packet new-packet-mark=other_up passthrough=no in-interface=LAN connection-mark=all_conn_other
11 ;;; other down
chain=prerouting action=mark-packet new-packet-mark=other_down passthrough=no in-interface=WAN connection-mark=all_conn_other
12 ;;; 256/128 upload
chain=forward action=mark-connection new-connection-mark=up_conn_256/128 passthrough=yes src-address-list=256/128
13 chain=forward action=mark-packet new-packet-mark=256/128_upload passthrough=yes
connection-mark=up_conn_256/128
14 ;;; 256/128 download
chain=forward action=mark-connection new-connection-mark=down_conn_256/128 passthrough=yes dst-address-list=256/128
15 chain=forward action=mark-packet new-packet-mark=256/128_download passthrough=yes connection-mark=down_conn_256/128


And Queues:

0 name=“DOWN_KOL” parent=global-in packet-mark=“” limit-at=950000 queue=sfq priority=4 max-limit=1024000 burst-limit=0 burst-threshold=0 burst-time=0s

1 name=“UP_KOL” parent=global-in packet-mark=“” limit-at=200000 queue=sfq priority=4 max-limit=256000 burst-limit=0 burst-threshold=0 burst-time=0s

2 name=“www_down_kol” parent=DOWN_KOL packet-mark=www_down limit-at=0 queue=download priority=2 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s

3 name=“www_upload_kol” parent=UP_KOL packet-mark=www_up limit-at=0 queue=upload priority=2 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s

4 name=“p2p_down_kol” parent=DOWN_KOL packet-mark=p2p_down limit-at=0 queue=download priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s

5 name=“p2p_upload_kol” parent=UP_KOL packet-mark=p2p_up limit-at=0 queue=upload priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s

6 name=“icmp_down_kol” parent=DOWN_KOL packet-mark=icmp_down limit-at=200000 queue=download priority=1 max-limit=256000 burst-limit=0 burst-threshold=0 burst-time=0s

7 name=“icmp_up_kol” parent=UP_KOL packet-mark=icmp_up limit-at=0 queue=upload priority=1 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s

8 name=“DOWN_klient” parent=global-out packet-mark=“” limit-at=0 queue=sfq priority=4 max-limit=1024000 burst-limit=0 burst-threshold=0 burst-time=0s

9 name=“UP_klient” parent=global-out packet-mark=“” limit-at=0 queue=sfq priority=4 max-limit=256000 burst-limit=0 burst-threshold=0 burst-time=0s

10 name=“256/128” parent=DOWN_klient packet-mark=256/128_download limit-at=0 queue=256_download priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s

11 name=“256/125” parent=UP_klient packet-mark=256/128_upload limit-at=0 queue=128_upload priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s

12 name=“other_down_kol” parent=DOWN_KOL packet-mark=other_down limit-at=0 queue=download priority=3 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s

13 name=“other_upload_kol” parent=UP_KOL packet-mark=other_up limit-at=0 queue=upload priority=3 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s

So problem is in in mangle as i know
But if i turn on the shape-marks the upload marks of priorities don’t work
After read everything what I found, i dont have idea what could be wrong… Maybe is somebody who could help me? What i need to read??

[galaxynet]

xordi -
I am not going to fix the whole setup but I am going ot point out a few things and hopeflly that and the docs you have handy will do the rest for you…

0 ;;; WWW
chain=prerouting action=mark-connection new-connection-mark=all_conn_www passthrough=yes protocol=tcp src-port=80,443

This will only mark ‘stuff’ that has a src-port of 80 or 443 - is that what you really want?

1 ;;; www up
chain=prerouting action=mark-packet new-packet-mark=www_up passthrough=no in-interface=LAN connection-mark=all_conn_www

This rule here will only mark the packet IF is has a connection mark of ‘all_conn_www’ AND it comes in the LAN interface. Otherwise it will not mark the packet and will fall through to the next rule.

2 ;;; www down
chain=prerouting action=mark-packet new-packet-mark=www_down passthrough=no in-interface=WAN connection-mark=all_conn_www

This rule will only mark packets that have a connection mark of ‘all_conn_www’ AND come in through the WAN interface, otherwise the packet falls through to the next rule. I am not really sure why you are trying to capture this one… If it is because you have a web server behind the router it is simpler to use the IP address to mangle by and then queue the bandwidth that way… If you are trying to ‘capture’ answers to client requests for web pages then read on…


Normally what you would do is this;

chain=prerouting action=mark-connection new-connection-mark=all_conn_www passthrough=yes protocol=tcp src-port=1024-65535 dst-port=80,443

You would normally also select an interface in the above rule like ‘in-interface=LAN’

or most use this to capture the connection as it passes THROUGH the router instead of when it ‘appears’ on a router interface…

chain=forward action=mark-connection new-connection-mark=all_conn_www passthrough=yes in-interface=LAN protocol=tcp src-port=1024-65535 dst-port=80,443
Either of these two rules will mark a connection whose destination is port 80 or 443. They also have the advantage of ‘capturing’ the return or ‘answer’ to the client request without further action from you.

Next -

chain=prerouting action=mark-packet new-packet-mark=www_up passthrough=no in-interface=LAN connection-mark=all_conn_www

OR

chain=forward action=mark-packet new-packet-mark=www_up passthrough=no in-interface=LAN connection-mark=all_conn_www

These mark the packets that match the connection paramenters set forth earlier, with the second one being the preferred method. There is no real need to use the ‘in-interface’ parameter here if you make that part of your original connection marking rule.

You have to chose which chain to mark connection/packets with and be consistant. There are times you want to use the prerouting chain, the forward chain, post-routing, input and output chains…you’ll need to do a little more reading to get a better handle on that.


Also looking at your mangle rules…you later go in to using the forward chain, where you change the connection mark and the packet marks to something other than what you had in the pre-routing chain. I can see why you may want to do it - not sure though if you understand what you are doing with that…

Maybe a little explanation is in order…

Pre-routing gets excuted before anything else happens. From there the connection/packet gets passed to either the input or forward chain. From there it goes to the either the output or post-routing chains.

They get excuted in numerical order (the number next to the rule is the numerical order).

Once a connection, a connection mark, packet or packet mark match a rule, and passthrough is set to ‘no’ then NO further processing in that chain will occur.



And to your queues -

0 name=“DOWN_KOL” parent=global-in packet-mark=“” limit-at=950000 queue=sfq priority=4 max-limit=1024000 burst-limit=0 burst-threshold=0 burst-time=0s

1 name=“UP_KOL” parent=global-in packet-mark=“” limit-at=200000 queue=sfq priority=4 max-limit=256000 burst-limit=0 burst-threshold=0 burst-time=0s

Normally one would be global-in and the other global-out… Or you could use the interface as the parent - LAN or WAN.

[bax]

In mangle Im also using prerouting and postrouting an it see that is ok , but I found (in some post) another way - to mark only forward … hm now I consider to test mangle forward packet .
What you guys thin which is better way to mark traffic ? prerouting&posrouting or just forward

[xordi]

Ok, lets start explaing…(sorry for english)
That what i want to do is:

• 1’st priority traffic from my LAN to WAN
• 2’nd shape the transfers of users on different hosts

For that think i need to mark all two times THE SAME traffic in two different ways (from LAN to WAN and from WAN to LAN), at 1’st i need mark the traffic for priorities in prerouting (inface=LAN for upload, inface=WAN for download because the prerouting is on each side but only inface is changing) and use it to priority in global-in (because global-in is on each side the same like prerouting only iface is changing) so i think i did it but something is wrong, maybe somebody explayn what?

at 2nd i need to mark packets in 2’nd chain-prerouting “to and from” each host (or from adress list), the same like in prerouting, only changing iface LAN (for upload) or WAN(for download)(??)… and use it to shape in global-out (because on each iface is global-out too)…

Can somebody tell me that is something wrong in my mind? and where i’m doing mistake?

Normally one would be global-in and the other global-out… Or you could use the interface as the parent - LAN or WAN

I cannot use it (global-in) for marks in forward because packet marks from prerouting “was” in global-in and they cannot appear second time in the same place… (i think)…

chain=forward action=mark-connection new-connection-mark=all_conn_www passthrough=yes in-interface=LAN protocol=tcp src-port=> 1024-65535 > dst-port=80,443

Either of these two rules will mark a connection whose destination is port 80 or 443. They also have the advantage of ‘capturing’ the return or ‘answer’ to the client request without further action from you.

I need to capture download and upload in different packet marks. (Thank you for the blu-coulored ports, i forgot about that )

Thats what i think about it… please tell me if i’m wrong.???

[galaxynet]

xordi -

Ok I have a little better understanding of what you are trying to do.

You do not have to separately mark ‘return’ traffic coming in the WAN interface. You marked the connection as it went through the router already. The only time you would want to mark traffic coming in on the WAN interface would be if you were shaping the bandwidth for a server behind the router. Because the server would not be starting the connections - it would answering the connections…

So without writing the rules for you here is how you go from selecting the type of traffic, then to possibly changing the marks basked on the client IP address.

So first mangle in the pre-routing chain the type of traffic, dst-port 80 in-interface=LAN
Give it a connection mark - set passthrough to yes.
Next while still in the pre-routing chain, mark the packet. Set passthrough to no.

Now we move on to the forward chain, use the connection mark you gave port 80 earlier. Start with connection-mark= (what ever it is you gave port 80 as a connection mark. While still on this rule go to the advanced tab, here you can select the src-addr list that has certain IPs that you want to shape differently. Select the list. Now make the action=mark-packet and give it the ‘new’ packet mark you want. Set passthrough=no.

So what happens here is that only the connections that have the port 80 connection mark AND have a specified IP address we change their packet mark. If either condition is false then the packet mark does not get changed and the connection continues through the forward chain to be evalutaed by all the rules until it either matches or runs out of rules…

Do you see now how we initially marked the connection and the packet. And then basically ‘filtered’ the connections again in the forward chain and changed their packet mark if we needed to. Effectively we were able to use the two chains to mark connections/packets as needed.

Once these connections are marked the connection mark (and packet marks) hold for the data as it travels both ways through the router. (LAN → WAN and LAN ← WAN). There is no need mark data as it comes in the WAN port unless you want to change the connection/packet marks.

Now under queuing well then maybe you need to use the interface (LAN / WAN) instead of global-in/out as the parent…

R/

Thom

[xordi]

But… Uhm… How to explayn…
I want to shape ALL packets from host/hosts in forward…
(Thats right thinking:
LAN > (mangle prerouting) > priorities in global-in > (mangle forward) > shaping in global-out > WAN

WAN > (mangle prerouting) > priorities in global-in > (mangle forward) > shaping in global-out < LAN

If i want to shape upload and download after mangle in prerouting in global-in i should have the direction (up or down) of packets and the same is in global-out

If i will have many connections(and mark packets) in prerouting for priorities … We should group all connections in forward (remarking) with src/dst host/hosts… and put packets marks I dont wanna shape only one type of connections … but ALL of connections(packets)

[galaxynet]

xordi -
You’re just not getting it. Let me try one last time…

Once you mark a connection ( LAN → WAN ) when the WAN source replies the related connection is also marked with the SAME connection mark.

As I explained above…you can mark any kind of traffic you want. I was showing how to mark the traffic type first. Then in the next chain (which is FORWARD) using the connection or packet mark to further change the packet or connection mark as needed - in your case there is a list of specific IPs that you wanted to shape differently that the all the others.

pre-routing mark packet wwwu traffic (in-interface LAN)
pre-routing mark packet ftpu traffic (in-interface LAN)
pre-routing mark packet sshu traffic (in-interface LAN)

forward - mark=wwwu, IP address on list? NO - then don’t change anything
. YES - then change packet mark

forward - mark=sshu, IP address on list? NO - then don’t change anything
. YES - then change packet mark

pre-routing mark packet wwwd traffic (in-interface WAN)
pre-routing mark packet ftpd traffic (in-interface WAN)
pre-routing mark packet sshd traffic (in-interface WAN)

forward - mark=wwwd, IP address on list? NO - then don’t change anything
. YES - then change packet mark

forward - mark=sshd, IP address on list? NO - then don’t change anything
. YES - then change packet mark

As to queuing, once again, when you set the queues packet marks and then place them under a parent they are ‘queued’. If you want to control what is coming in/going out a particular interface, then use the interface and not the global ‘queues’.

parent-upload=LAN
parent-dnload=WAN

queue 1, packet mark=wwwu parent=upload max-limit=256k

queue 7, packet mark=wwwd parent=dnload max-limit=128

[xordi]

http://forum.mikrotik.com/t/is-priority-really-working/24459/1

In this post is my thinking, thats what i want to do, shame that i foud this post right now… Thank you for Your help

That’s great, somebody is for helping on this forum. Not only RTFMM and RTFMM

Thank You mate! If you or somebody want i can put here my further configs for analisess.