QoS using mangle rules... please check my work!

nickb · Mon Nov 15, 2010 9:11 pm

Greetings all,

I've used the examples from Janis Megis' MUM presentation (http://mum.mikrotik.com/presentations/C ... _Megis.pdf) to create queue-tree bandwidth shaping for my customers quite some time ago, and it has been working well.

Today, I've added in packet prioritization - but the last three pages of the PDF are pretty sketchy in how to actually implement this feature; so I've taken my best guess.

I am hoping that somebody more knowledgable than myself with QoS/Mangle/Queue Trees can look at this and tell me if I've done it properly or not!

Here is my "mangle" configuration:

 /ip firewall mangle
add action=mark-packet chain=prerouting comment="Mark p2p services" disabled=\
    no new-packet-mark=P2P_services p2p=all-p2p passthrough=yes
add action=mark-packet chain=prerouting comment=\
    "Download Services (no packet quantifiers)" disabled=no dst-port=\
    110,995,143,993,25,20,21 new-packet-mark=Download_services passthrough=\
    yes protocol=tcp
add action=mark-packet chain=prerouting comment="http download" \
    connection-bytes=5000000-0 disabled=no dst-port=80 new-packet-mark=\
    Download_services passthrough=yes protocol=tcp
add action=mark-packet chain=prerouting comment="http requests" \
    connection-bytes=0-500000 disabled=no dst-port=80 new-packet-mark=\
    Ensign_services passthrough=yes protocol=tcp
add action=mark-packet chain=prerouting comment="SFTP Transfers" disabled=no \
    dst-port=22 new-packet-mark=Download_services packet-size=1400-1500 \
    passthrough=yes protocol=tcp
add action=mark-packet chain=prerouting comment=\
    "TCP Ensign Services (no packet quantifiers)" disabled=no dst-port=\
    53,443,23 new-packet-mark=Ensign_services passthrough=yes protocol=tcp
add action=mark-packet chain=prerouting comment=\
    "UDP Ensign Services (no packet quantifiers)" disabled=no dst-port=53 \
    new-packet-mark=Ensign_services passthrough=yes protocol=udp
add action=mark-packet chain=prerouting comment="ICMP (ensign)" disabled=no \
    new-packet-mark=Ensign_services passthrough=yes protocol=icmp
add action=mark-packet chain=prerouting comment="SSH Shell" disabled=no \
    dst-port=22 new-packet-mark=Ensign_services packet-size=0-1400 \
    passthrough=yes protocol=tcp
add action=set-priority chain=forward comment=\
    "Set Ensign_services to Priority 1" disabled=no new-priority=1 \
    packet-mark=Ensign_services passthrough=yes
add action=set-priority chain=forward comment=\
    "Set download services to priority 7" disabled=no new-priority=7 \
    packet-mark=Download_services passthrough=yes
add action=set-priority chain=forward comment=\
    "Set p2p services to priority 8" disabled=no new-priority=8 packet-mark=\
    P2P_services passthrough=yes
add action=mark-connection chain=prerouting comment="Mark Restricted Traffic" \
    disabled=no dst-address-list=Restricted in-interface=Feed \
    new-connection-mark=Restricted_Connection_DL passthrough=yes
add action=mark-connection chain=postrouting comment="" disabled=no \
    new-connection-mark=Restricted_Connection_UL out-interface=Feed \
    passthrough=yes src-address-list=Restricted
add action=mark-packet chain=prerouting comment="" connection-mark=\
    Restricted_Connection disabled=no dst-address-list=Restricted \
    in-interface=Feed new-packet-mark=Restricted_Traffic_DL passthrough=no
add action=mark-packet chain=postrouting comment="" connection-mark=\
    Restricted_Traffic_UL disabled=no new-packet-mark=Restricted_Traffic_UL \
    out-interface=Feed passthrough=no src-address-list=Restricted
add action=mark-connection chain=prerouting comment="Mark Standard Traffic" \
    disabled=no dst-address-list=Standard in-interface=Feed \
    new-connection-mark=Standard_Connection_DL passthrough=yes
add action=mark-connection chain=postrouting comment="" disabled=no \
    new-connection-mark=Standard_Connection_UL out-interface=Feed \
    passthrough=yes src-address-list=Standard
add action=mark-packet chain=prerouting comment="" connection-mark=\
    Standard_Connection_DL disabled=no dst-address-list=Standard \
    in-interface=Feed new-packet-mark=Standard_Traffic_DL passthrough=no
add action=mark-packet chain=postrouting comment="" connection-mark=\
    Standard_Connection_UL disabled=no new-packet-mark=Standard_Traffic_UL \
    out-interface=Feed passthrough=no src-address-list=Standard
add action=mark-connection chain=prerouting comment="Mark Enhanced Traffic" \
    disabled=no in-interface=Feed new-connection-mark=Enhanced_Connection_DL \
    passthrough=yes src-address-list=Enhanced
add action=mark-connection chain=postrouting comment="" disabled=no \
    new-connection-mark=Enhanced_Connection_UL out-interface=Feed \
    passthrough=yes src-address-list=Enhanced
add action=mark-packet chain=prerouting comment="" connection-mark=\
    Enhanced_Connection_DL disabled=no dst-address-list=Enhanced \
    in-interface=Feed new-packet-mark=Enhanced_Traffic_DL passthrough=no
add action=mark-packet chain=postrouting comment="" connection-mark=\
    Enhanced_Connection_UL disabled=no new-packet-mark=Enhanced_Traffic_UL \
    out-interface=Feed passthrough=no src-address-list=Enhanced
add action=log chain=forward comment="Check for unmarked traffic" disabled=\
    yes log-prefix=""

Queue Tree:

/queue tree
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 \
    max-limit=0 name=Total_Download parent=global-in priority=8
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 \
    max-limit=0 name=Total_Upload parent=global-out priority=8
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 \
    max-limit=0 name=Restricted_Download packet-mark=Restricted_Traffic_DL \
    parent=Total_Download priority=8 queue=PCQ_down_768k
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 \
    max-limit=0 name=Standard_Download packet-mark=Standard_Traffic_DL \
    parent=Total_Download priority=4 queue=PCQ_down_1M
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 \
    max-limit=0 name=Enhanced_Download packet-mark=Enhanced_Traffic_DL \
    parent=Total_Download priority=2 queue=PCQ_down_2M
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 \
    max-limit=0 name=Restricted_Upload packet-mark=Restricted_Traffic_UL \
    parent=Total_Upload priority=8 queue=PCQ_up_256k
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 \
    max-limit=0 name=Standard_Upload packet-mark=Standard_Traffic_UL parent=\
    Total_Upload priority=4 queue=PCQ_up_384k
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 \
    max-limit=0 name=Enhanced_Upload packet-mark=Enhanced_Traffic_UL parent=\
    Total_Upload priority=4 queue=PCQ_up_512k

Queue Types:

/queue type
set default kind=pfifo name=default pfifo-limit=50
set ethernet-default kind=pfifo name=ethernet-default pfifo-limit=50
set wireless-default kind=sfq name=wireless-default sfq-allot=1514 \
    sfq-perturb=5
set synchronous-default kind=red name=synchronous-default red-avg-packet=1000 \
    red-burst=20 red-limit=60 red-max-threshold=50 red-min-threshold=10
set hotspot-default kind=sfq name=hotspot-default sfq-allot=1514 sfq-perturb=\
    5
add kind=pcq name=pcq-download pcq-classifier=dst-address pcq-limit=50 \
    pcq-rate=0 pcq-total-limit=2000
add kind=pcq name=pcq-upload pcq-classifier=src-address pcq-limit=50 \
    pcq-rate=0 pcq-total-limit=2000
add kind=pcq name=PCQ_down_2M pcq-classifier=dst-address pcq-limit=20 \
    pcq-rate=2000000 pcq-total-limit=500
add kind=pcq name=PCQ_up_512k pcq-classifier=src-address pcq-limit=20 \
    pcq-rate=512000 pcq-total-limit=500
add kind=pcq name=PCQ_down_1M pcq-classifier=dst-address pcq-limit=30 \
    pcq-rate=1000000 pcq-total-limit=1000
add kind=pcq name=PCQ_down_768k pcq-classifier=dst-address pcq-limit=30 \
    pcq-rate=768000 pcq-total-limit=1000
add kind=pcq name=PCQ_up_384k pcq-classifier=src-address pcq-limit=30 \
    pcq-rate=384000 pcq-total-limit=1000
add kind=pcq name=PCQ_up_256k pcq-classifier=src-address pcq-limit=30 \
    pcq-rate=256000 pcq-total-limit=1000
set default-small kind=pfifo name=default-small pfifo-limit=10

The bandwidth limiting parts for the "Restricted", "Standard", and "Enhanced" traffic seems to work well.

What I am really curious about is if what I've done with 'QoS' is correct? Basically I've followed the second to last page in Megis' document to create mangle rules in the PREROUTING chain to mark the packets; then used mangle rules on the FORWARD chain.

This screen shot indicates what's happening with the bytes/packet counts with these rules. Note that this is in the middle of the day and these are all residential customers, so traffic is pretty low:

mangle.PNG

This screen shot shows my queue configuration visually:

Queue.PNG

I would appreciate any comments as to this configuration and any suggested changes.

Thank you!

prince90s · Sat Jan 29, 2011 9:15 pm

Wow, it looks great, I'll try first, then give you advice. I hope I can communicate with you about ROS, my MSN is jq1018@live.cn.

benmikrotik · Tue Dec 11, 2012 1:24 am

Can hotspot authentication work with Megis QoS?
How about user bandwidth control conflict with hotspot?

_saik0 · Sun Mar 10, 2013 2:46 pm

Can you explain those "no packet quantifiers" comments?

I'm also following the Megis approach but apparently traffic/packet counters for prerouting mangle rules don't work when forwarding rules for same traffic are applied.
Can the prerouting marked packets in global-in HTB still get prioritised properly even when those counters in prerouting mangle suggest no traffic is seen?

nickb · Mon Mar 11, 2013 5:13 pm

Can hotspot authentication work with Megis QoS?
How about user bandwidth control conflict with hotspot?

I haven't tried, but it seems like it should be possible. If you don't set a hotspot bandwidth setting in the hotspot profile, then set an address-list for your hotspot subnet to assign it to a queue leaf, that should work.

Can you explain those "no packet quantifiers" comments?

I'm also following the Megis approach but apparently traffic/packet counters for prerouting mangle rules don't work when forwarding rules for same traffic are applied.
Can the prerouting marked packets in global-in HTB still get prioritised properly even when those counters in prerouting mangle suggest no traffic is seen?

"no packet quantifiers" means that it's not counting any packets.

If you look at the next two rules "http download" and "http request", the packets are quantified (counted) in that 0-500,000 bytes is considered to be the "http request", then 500,000+ is considered "download", and prioritized differently.

_saik0 · Mon Mar 11, 2013 5:27 pm

Then you could possibly help with the issue I have.

Like I said I'm following the Megis approach.
I tried to setup QoS in 2 steps: 1. prioritisation (prerouting packet mangling - global-in HTB), 2. PCQ (forward packet mangling - global-out HTB)

I'm experiencing the issue where prerouting rules don't see all the traffic (judging by byte/packet counters on the rule itself) when forward rules are applied to the same traffic. Immediately, when I disable the forward rules, prerouting rules start seeing all the traffic.

I'm not talking about the marking as the marking of the prerouting mangle would be overwritten by forward mangle, that i know.
But how come the packet/byte counters are affected?
Is this relevant to the QT/HTB at all?
I mean, I'm not getting the wanted result from the QT/HTB so my first thought is that mangling doesn't work properly thus prioritisation can't work.

If the same type of traffic is first market in prerouting, then later in forward, how should the packet/byte counters for said rules behave? Would only the counters on rule in forward increase or both in forward and in prerouting?

Maybe if you could take a look at the thread I started: http://forum.mikrotik.com/viewtopic.php?f=2&t=70617

Thanks

QoS using mangle rules... please check my work!

QoS using mangle rules... please check my work!

Re: QoS using mangle rules... please check my work!

Re: QoS using mangle rules... please check my work!

Re: QoS using mangle rules... please check my work!

Re: QoS using mangle rules... please check my work!

Re: QoS using mangle rules... please check my work!

Who is online