Double marking and duo Queue tree, which interfaces?

I use mum/wiki example: http://mum.mikrotik.com/presentations/CZ09/QoS_Megis.pdf

I have Asymmetrical Internet, low upload, high download capacity.

I mangle in prerouting for service packer marker.
I mangle in forward client traffic based on address list and subscription level.

In Queue tree have priority and speed limits set for services:
Upload; Parent = Public interface
Download; Parent = global-in

In Queue tree have priority and speed limits set for client subscripiton levels:
Upload; Parent = ???
Download; Parent = Local interface.

I tried several options for client upload parent; global out, global total or Publick interface again. Nothing works.

MUM presentation only speaks about global-in for download prioritizing and limiting for service QoS. But I have very little upload capacity and since P2P is consuming lots of bandwith, need to prioritize and limit in upload as well, but with different limits as download. Which interface (real or virtual) to use?
Normally I would say the public interface. But that one is already in use by the client shaping Queues. Can anybody give me any help?

this is my Queue tree:

/queue tree
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=1200k name=upload1 packet-mark=“” parent=ether1
priority=4
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=200k max-limit=700k name=“U-High Priority TrafficU” packet-mark=
U_HIGH parent=upload1 priority=1 queue=pcq_up
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=1100k max-limit=1200k name=“High Priority TrafficU” packet-mark=
HIGH parent=upload1 priority=3 queue=pcq_up
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=200k max-limit=1M name=“Med Priority TrafficU” packet-mark=MED
parent=upload1 priority=5 queue=pcq_up
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=50k max-limit=400k name=“Low Priority TrafficU” packet-mark=LOW
parent=upload1 priority=8 queue=pcq_up_LP
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=18M name=download1 packet-mark=“” parent=global-in
priority=5
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=3M max-limit=17M name=“High Priority Traffic” packet-mark=HIGH
parent=download1 priority=3 queue=pcq_dwn
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=10M max-limit=18M name=“Med Priority Traffic” packet-mark=MED
parent=download1 priority=5 queue=pcq_dwn
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=3M max-limit=18M name=“Low Priority Traffic” packet-mark=LOW
parent=download1 priority=8 queue=pcq_dwn
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=2M max-limit=6M name=“U-High Priority Traffic” packet-mark=U_HIGH
parent=download1 priority=1 queue=pcq_dwn
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=50k max-limit=600k name=“P2P TrafficU” packet-mark=P2P parent=
upload1 priority=7 queue=pcq_up_LP
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=3M max-limit=18M name=“P2P Traffic” packet-mark=P2P parent=
download1 priority=7 queue=pcq_dwn
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=21M name=Total_download parent=“MC-bridge (E2+3)”
priority=2
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=6M max-limit=21M name=Comfort_client_download packet-mark=
Comfort_Packet parent=Total_download priority=5 queue=PCQ_down_4M
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=1600k name=Total_upload parent=ether1 priority=1
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=256k max-limit=1600k name=Comfort_client_upload packet-mark=
Comfort_Packet parent=Total_upload priority=5 queue=PCQ_up_256k
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=1M max-limit=21M name=Basic_client_download packet-mark=
Basic_Packet parent=Total_download priority=6 queue=PCQ_down_2M
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=512k max-limit=10M name=Holiday_client_download packet-mark=
HH_Packet parent=Total_download priority=7 queue=PCQ_down_1M
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=6M max-limit=21M name=Platinum_client_download packet-mark=
Platinum_Packet parent=Total_download priority=3 queue=PCQ_down_8M
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=8M max-limit=21M name=Business_client_download packet-mark=
Bussiness_Packet parent=Total_download priority=2 queue=PCQ_down_5M
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=512k max-limit=21M name=Manager_download packet-mark=Manager_Packet
parent=Total_download priority=1 queue=pcq_dwn
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=128k max-limit=1600k name=Manager_upload packet-mark=Manager_Packet
parent=Total_upload priority=1 queue=pcq_up
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=512k max-limit=1600k name=Business_client_upload packet-mark=
Bussiness_Packet parent=Total_upload priority=2 queue=PCQ_up_1M
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=320k max-limit=1600k name=Platinum_client_upload packet-mark=
Platinum_Packet parent=Total_upload priority=3 queue=PCQ_up_256k
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=256k max-limit=1600k name=Basic_client_upload packet-mark=
Basic_Packet parent=Total_upload priority=6 queue=PCQ_up_128k
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=128k max-limit=1600k name=Holiday_client_upload packet-mark=
HH_Packet parent=Total_upload priority=8 queue=PCQ_up_96k


This setup is not working. So changes are needed. Anybody that can give some advice?

Hi,

I also use double mark and my config is:
For services:
Upload; Parent = global-in
Download; Parent = global-in

For client subscripiton levels:
Upload; Parent = Public interface (WAN)
Download; Parent = Local interface (LAN)

My mangle is similar to yours.
Hope this help.

Regards, G.

WirelessRudy, do you use NAT? actually, for such complex setups I prefer not to use interface queues…

you may use global-in for user upload and download per service, and global-out for user download and upload per service

When I do that my upload queues drop dead. No more traffic.

For client subscripiton levels:
Upload; Parent = Public interface (WAN)
Download; Parent = Local interface (LAN)

When I do that my upload queues also bear no traffic

My mangle is similar to yours.
Hope this help.

Regards, G.

If you don’t mind, can you post your config (with the pcq’s you’re using) here?
If you do mind, would you be kind enough to send them to my e-mail info@marucom.es?
I would appreciate it “mucho” :smiley:

No, no NAT in this router. (So this router passes all traffic with original client CPE IP and port. Nat is only performed at adsl modems outside the WAN of my load balancer and on the WAN ports of the Load balancer itself.)

actually, for such complex setups I prefer not to use interface queues…

Why? I am learned from guys like galaxynet and other examples this is a good idea? Can you give me a funded reason? Maybe I agree with you! I am eager to see any input in improving my scheme.


you may use global-in for user upload and download per service, and global-out for user download and upload per service

Tried that. Although my service queues seem to be working now only the download queue on the users gets traffic, but much less then the service queues for download. (Should be roughly the same? Same amount of agregated sum of all services data should be the same as aggregated sum of all users data?)

Upload queues for the user stay empty. No traffic…

Well, I set my service upload to global-out as suggested but traffic was then doubled and some child and parent continuously in red. Internet service became crap…
I had to sad my queues back to origin (only service queues with download>global-in and upload>public interface) at least that works and worked for months…

Don’t know where to go now… I am thinking of having just another router in the pipe and split the dual functionality over the two routers. I might then also do route marking in mangle for policy routing of P2P or other unwanted traffic to my adsl lines since they are installing my HQ line this week.

Just got an answer form MT-support about the same question as this topic. Answer = “watch our tiktube with the QoS presentation”.
Gosh, sometimes I hate these guys. As if I not already have seen that presentation about 100 times…
:frowning:

Ok, I think where to find my problem and the solution..

Someone told me in the past to mangle only once, in the prerouting (or forward) chain on the local interface. (Where all traffic is coming from.
But I should actually mangle twice. (Actually three times. 1. mangle conn. 2. mangle upload 3. mangle download)
First I mangle all connections in pre-routing and assign connection mark.
Second these now known connections both get packet marked with separation based on the interface they come in.
Upload traffic come in the local interface.
Download traffic was initiated in the local interface (and conn.tracker started tracking it) but only when the result traffic comes back it comes in the public interface. Conn.tracker knows which connection it is, thus connection mark is applied already and now we only mark these packages on this incoming interface?

So far so good?

The queueing is now done for each direction based on the global-out interface? (If this is not true, please explain.)
[Wiki manual writes; "global-out: represents all the output interfaces in general (EGRESS queue). "

I don’t understand why actually most of us (including me up to now) use the global-in interface for traffic that leaves the router? Can somebody explain?

Hi,

According to Wiki:
global-in: represents all the input interfaces in general (INGRESS queue). Queues attached to global-in apply to traffic that is received by the router before the packet filtering
And look at packet flow diagram http://wiki.mikrotik.com/wiki/Packet_Flow
prerouting.JPG
When you mangle on prerouting, global-in is after mangle, so it works.

Regards, Grzegorz.

Ok, I agree on that. For confirmation, lets follow the traffic:
LAN client sends request for webpage > enters router at local interface and after conn tracking register the connection it gets mangled in the pre-routing chain with classifier that local interface. Here-after the packages are queued in the global-in.
After this traffic flows further through the router and leaves via public interface of router.

Then traffic comes back, and enters router at the public interface. Yet again the connection tracker is passed first and here the router recognizes the connection as one already created before. So same connection marker is applied. Now mangle should mark that connection again but the argument is now the public interface as incoming interface and a packet marker is applied that represents the download traffic.
Queued can now be done again in global-in, but we take the download package marker now as argument?

Would this now reserve the forward mangle for client (IP based) traffic shaping and the queue for that takes place in either the real output interface or the virtual one, Global-Out?

I know this might look like I did not do enough reading and testing. But I also post since I know lots of users are struggling with the subject. So to benefit us all! :smiley:

Little bit different item, but still QoS. More mangle this time.

In examples and Wiki I see two mainstream ways of mangle marking traffic flows.

Some use connection marking first, different traffic stream identifiers (filters) mark these connections with one connection marker “X”.
After this package marker rule marks all packages belonging to that connection with a certain package mark.

It is stated somewhere this is less cpu expensive since package belonging to one connection flow is now marked all in one packet marker rule. Without a marked connection router would have to filter and test each individual package before decision can be made what package mark it would get. And that for each package in a tream over and over again.

But some in this forum I see only marking packages. No use of connection marking.

Who is right?

Also, looking at my previous post in this topic, I only have to mark connection once. Even traffic coming back and entering public interface is immediately recognized and needs no filter. It just gets a new package mark if we tell router to do so.
Saves a whole bunch of filter rules I would think?

Depends on how often and what you’re matching. To mark a packet matching on a connection-mark takes one lookup per packet (does it have this tag?). If you’re marking the connection based on more than one matcher (does it match this source address, and this inbound interface) you’re saving resources since the lookups per packet go down, you only need to compare one field on every packet instead of two.

However, if you’re just matching on one matcher (did it come in through this interface?) and you don’t need to be stateful, you might as well just mark the packet.

Thanks, this is a clear explanation someone can understand. :slight_smile: :slight_smile:
Since most of my mangle filters look at at least two types of matcher and some whole ranges of ports (VOIP: 5060-5070) my choice for conn.marking is a right one. :slight_smile:

One question: What exactly do you mean with “stateful”? English is not my native language and my dictionary doesn’t even know that word…

A stateful firewall keeps track of connections (it keeps state over the packets involved in a connection).
http://en.wikipedia.org/wiki/Stateful_firewall

Thanks, very helpful.
running conn. tracker makes router ‘stateful’. That’s what I need.

I have a problem with double marking. When the upload Services Queue is active, the Users upload Queue are not counted!
But when Upload Services is disabled, then works well…

Here is My code (I have a NAT):

My Mangle:

;;; Download services
chain=prerouting action=mark-packet new-packet-mark=download_services_download passthrough=no protocol=tcp 
in-interface=bridge_ext src-port=110,995,143,993,25,20025 
chain=postrouting action=mark-packet new-packet-mark=download_services_upload passthrough=no protocol=tcp 
out-interface=bridge_ext dst-port=110,995,143,993,25,20025 

chain=prerouting action=mark-packet new-packet-mark=download_services_download passthrough=no protocol=tcp 
in-interface=bridge_ext src-port=80 connection-bytes=500000-0 
chain=postrouting action=mark-packet new-packet-mark=download_services_upload passthrough=no protocol=tcp 
out-interface=bridge_ext dst-port=80 connection-bytes=500000-0 

chain=prerouting action=mark-packet new-packet-mark=download_services_download passthrough=no protocol=tcp 
in-interface=bridge_ext src-port=20,21
chain=postrouting action=mark-packet new-packet-mark=download_services_upload passthrough=no protocol=tcp 
out-interface=bridge_ext dst-port=20,21 packet-size=0 

chain=prerouting action=mark-packet new-packet-mark=ensign_services_download passthrough=no protocol=tcp 
in-interface=bridge_ext src-port=53,37 
chain=postrouting action=mark-packet new-packet-mark=ensign_services_upload passthrough=no protocol=tcp 
out-interface=bridge_ext dst-port=53,37 

chain=prerouting action=mark-packet new-packet-mark=ensign_services_download passthrough=no protocol=udp 
in-interface=bridge_ext src-port=53,88,37 
chain=postrouting action=mark-packet new-packet-mark=ensign_services_upload passthrough=no protocol=udp 
out-interface=bridge_ext dst-port=53,88,37 

chain=prerouting action=mark-packet new-packet-mark=ensign_services_download passthrough=no protocol=icmp 
in-interface=bridge_ext
chain=postrouting action=mark-packet new-packet-mark=ensign_services_upload passthrough=no protocol=icmp 
out-interface=bridge_ext

chain=prerouting action=mark-packet new-packet-mark=ensign_services_download passthrough=no protocol=tcp 
in-interface=bridge_ext src-port=443 connection-bytes=0-500000 
chain=postrouting action=mark-packet new-packet-mark=ensign_services_upload passthrough=no protocol=tcp 
out-interface=bridge_ext dst-port=443 

chain=prerouting action=mark-packet new-packet-mark=ensign_services_download passthrough=no protocol=tcp 
in-interface=bridge_ext src-port=23 
chain=postrouting action=mark-packet new-packet-mark=ensign_services_upload passthrough=no protocol=tcp 
out-interface=bridge_ext dst-port=23 

chain=prerouting action=mark-packet new-packet-mark=ensign_services_download passthrough=no protocol=tcp 
in-interface=bridge_ext src-port=22,10203
chain=postrouting action=mark-packet new-packet-mark=ensign_services_upload passthrough=no protocol=tcp 
out-interface=bridge_ext dst-port=22,10203 

chain=prerouting action=mark-packet new-packet-mark=ensign_services_download passthrough=no protocol=tcp 
src-address-list="" in-interface=bridge_ext src-port=80 connection-bytes=0-500000 
chain=postrouting action=mark-packet new-packet-mark=ensign_services_upload passthrough=no protocol=tcp 
src-address-list="" out-interface=bridge_ext dst-port=80 connection-bytes=0-500000 

chain=prerouting action=mark-packet new-packet-mark=p2p_services_download passthrough=no p2p=all-p2p in-interface=bridge_ext 
chain=postrouting action=mark-packet new-packet-mark=p2p_services_upload passthrough=no p2p=all-p2p out-interface=bridge_ext 

chain=prerouting action=mark-packet new-packet-mark=other_services_download passthrough=no protocol=tcp in-interface=bridge_ext
chain=postrouting action=mark-packet new-packet-mark=other_services_upload passthrough=no protocol=tcp out-interface=bridge_ext 

chain=prerouting action=mark-packet new-packet-mark=other_services_download passthrough=no protocol=udp in-interface=bridge_ext 
chain=postrouting action=mark-packet new-packet-mark=other_services_upload passthrough=no protocol=udp out-interface=bridge_ext 

chain=prerouting action=mark-packet new-packet-mark=other_services_download passthrough=no in-interface=bridge_ext 
chain=postrouting action=mark-packet new-packet-mark=other_services_upload passthrough=no out-interface=bridge_ext 


;;; mark basic client traffic
chain=forward action=mark-connection new-connection-mark=basic_client_conn passthrough=yes src-address-list=Basic_class_client
chain=forward action=mark-packet new-packet-mark=basic_client_traffic passthrough=yes connection-mark=basic_client_conn 

;;; mark standard client traffic
chain=forward action=mark-connection new-connection-mark=standard_client_conn passthrough=yes src-address-list=Standard_class_client 
chain=forward action=mark-packet new-packet-mark=standard_client_traffic passthrough=yes connection-mark=standard_client_conn 

;;; mark business client traffic
chain=forward action=mark-connection new-connection-mark=business_client_conn passthrough=yes src-address-list=Business_class_client 
chain=forward action=mark-packet new-packet-mark=business_client_traffic passthrough=yes connection-mark=business_client_conn 

X ;;; Check for unmarked traffic chain=forward action=log log-prefix=""

My Queue Tree:

# CLIENTS Download
name="Total_download_clients" parent=global-out limit-at=0 priority=1 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s 

name="basic_client_download" parent=Total_download_clients packet-mark=basic_client_traffic limit-at=0 queue=PCQ_down_375k 
priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s 

name="standard_client_download" parent=Total_download_clients packet-mark=standard_client_traffic limit-at=0 queue=PCQ_down_750k 
priority=4 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s 

name="business_client_download" parent=Total_download_clients packet-mark=business_client_traffic limit-at=0 queue=PCQ_down_2M 
priority=1 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s 

# CLIENTS Upload
name="Total_upload_clients" parent=global-in packet-mark="" limit-at=0 priority=1 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s 

name="basic_client_upload" parent=Total_upload_clients packet-mark=basic_client_traffic limit-at=0 queue=PCQ_up_128k priority=8 
max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s 

name="standard_client_upload" parent=Total_upload_clients packet-mark=standard_client_traffic limit-at=0 queue=PCQ_up_250k 
priority=4 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s 

name="business_client_upload" parent=Total_upload_clients packet-mark=business_client_traffic limit-at=0 queue=PCQ_up_1M 
priority=1 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s 


# SERVICES Download
name="Total_download_services" parent=global-in limit-at=0 priority=1 max-limit=4M burst-limit=0 burst-threshold=0 burst-time=0s 

name="Ensign_services_download" parent=Total_download_services packet-mark=ensign_services_download limit-at=0 queue=default priority=1 
max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s 

name="User_requests_download" parent=Total_download_services limit-at=0 queue=default priority=3 max-limit=0 burst-limit=0 burst-threshold=0 
 burst-time=0s 

name="Communication_services_download" parent=Total_download_services limit-at=0 queue=default priority=5 max-limit=0 burst-limit=0 
burst-threshold=0 burst-time=0s

name="Download_services_download" parent=Total_download_services packet-mark=download_services_download limit-at=0 queue=default 
priority=7 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s 

name="P2P_services_download" parent=Total_download_services packet-mark=p2p_services_download limit-at=0 queue=default priority=8 
max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s 

name="Other_services_download" parent=Total_download_services packet-mark=other_services_download limit-at=0 queue=default priority=7 
max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s 

# SERVICES Upload
name="Total_upload_services" parent=global-out limit-at=0 priority=1 max-limit=2M burst-limit=0 burst-threshold=0 burst-time=0s 

name="Download_services_upload" parent=Total_upload_services packet-mark=download_services_upload limit-at=0 queue=default priority=7 
max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s

name="Ensign_services_upload" parent=Total_upload_services packet-mark=ensign_services_upload limit-at=0 queue=default priority=1 
max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s 

name="P2P_services_upload" parent=Total_upload_services packet-mark=p2p_services_upload limit-at=0 queue=default priority=8 
max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s 

name="Communication_services_upload" parent=Total_upload_services limit-at=0 queue=default priority=5 max-limit=0 burst-limit=0 
burst-threshold=0 burst-time=0s 

name="User_requests_upload" parent=Total_upload_services limit-at=0 queue=default priority=3 max-limit=0 burst-limit=0 burst-threshold=0 
burst-time=0s 

name="Other_services_upload" parent=Total_upload_services packet-mark=other_services_upload limit-at=0 queue=default priority=7 
max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s

How can I fix it?

Anyone?

it’s because you’re marking (both upload and download) in Forward, and Forward is after Global-in (where your upload queues are)

Thanks! What would You recommend?
Should be removed upload marking from Forward?

you can move your upload queues to global-out - just change marking rules so that they use different marks for upload and download