Packet Mark issues w/ queues

RouterOS Beginner Basics
1

Hello, I’m having issues with packet marks. as in, they’re not marking packets.

Currently i have rule setup to mark any packet from the ip address 192.168.135.199 with 768d.
and I have a queue setup to limit anything with a packet mark of 768d to 768k.
My problem is when i start downloading, the packet mark counter doesn’t do much at all, downloaded over 400mb and it says its marked less then 16mb. needless to say, the download wasn’t limited.

(side note, we use PCC to balence 4 DSL connections, with some rules to send some traffic from some hosts over a static DSL line)

Here’s my cfg.

TIA

Mangle

/ip firewall mangle
add action=mark-connection chain=input comment="" disabled=no in-interface=\
    pppoe-out1 new-connection-mark=wan1_conn passthrough=yes
add action=mark-connection chain=input comment="" disabled=no in-interface=\
    pppoe-out2 new-connection-mark=wan2_conn passthrough=yes
add action=mark-connection chain=input comment="" disabled=no in-interface=\
    pppoe-out3 new-connection-mark=wan3_conn passthrough=yes
add action=mark-connection chain=input comment="" disabled=no in-interface=\
    pppoe-out4 new-connection-mark=wan4_conn passthrough=yes
add action=mark-routing chain=output comment="" connection-mark=wan1_conn \
    disabled=no new-routing-mark=to_wan1 passthrough=yes
add action=mark-routing chain=output comment="" connection-mark=wan2_conn \
    disabled=no new-routing-mark=to_wan2 passthrough=yes
add action=mark-routing chain=output comment="" connection-mark=wan3_conn \
    disabled=no new-routing-mark=to_wan3 passthrough=yes
add action=mark-routing chain=output comment="" connection-mark=wan4_conn \
    disabled=no new-routing-mark=to_wan4 passthrough=yes
add action=mark-connection chain=prerouting comment="" disabled=no \
    dst-address-type=!local in-interface=ether1 new-connection-mark=wan1_conn \
    passthrough=yes per-connection-classifier=both-addresses-and-ports:4/0
add action=mark-connection chain=prerouting comment="" disabled=no \
    dst-address-type=!local in-interface=ether1 new-connection-mark=wan2_conn \
    passthrough=yes per-connection-classifier=both-addresses-and-ports:4/1
add action=mark-connection chain=prerouting comment="" disabled=no \
    dst-address-type=!local in-interface=ether1 new-connection-mark=wan3_conn \
    passthrough=yes per-connection-classifier=both-addresses-and-ports:4/2
add action=mark-connection chain=prerouting comment="" disabled=no \
    dst-address-type=!local in-interface=ether1 new-connection-mark=wan4_conn \
    passthrough=yes per-connection-classifier=both-addresses-and-ports:4/3
add action=mark-routing chain=prerouting comment="" connection-mark=wan1_conn \
    disabled=no in-interface=ether1 new-routing-mark=to_wan1 passthrough=yes
add action=mark-routing chain=prerouting comment="" connection-mark=wan2_conn \
    disabled=no in-interface=ether1 new-routing-mark=to_wan2 passthrough=yes
add action=mark-routing chain=prerouting comment="" connection-mark=wan3_conn \
    disabled=no in-interface=ether1 new-routing-mark=to_wan3 passthrough=yes
add action=mark-routing chain=prerouting comment="" connection-mark=wan4_conn \
    disabled=no in-interface=ether1 new-routing-mark=to_wan4 passthrough=yes
add action=mark-packet chain=prerouting comment="User 768D" disabled=no \
    new-packet-mark=768d passthrough=no src-address=192.168.135.199
add action=mark-connection chain=prerouting comment=Test disabled=no \
    dst-port=8080 in-interface=ether9 new-connection-mark=SSL2 passthrough=\
    yes protocol=tcp
add action=mark-connection chain=prerouting comment=\
    "Moves SSL connections to PPPoE_1" disabled=no dst-port=443 in-interface=\
    ether1 new-connection-mark=SSL passthrough=yes protocol=tcp
add action=mark-connection chain=prerouting comment=\
    "For Jan Tidus' Internet banking. Marks Connection" disabled=no \
    in-interface=ether1 new-connection-mark=SSL passthrough=yes \
    src-mac-address=00:15:6D:F8:18:11
add action=mark-connection chain=prerouting comment=\
    "Burns Mark Connection -> SSL" disabled=no in-interface=ether1 \
    new-connection-mark=SSL passthrough=yes src-mac-address=00:15:6D:FA:D7:51
add action=mark-connection chain=prerouting comment=Chophouse disabled=no \
    new-connection-mark=LocksMill passthrough=yes src-mac-address=\
    00:15:6D:FA:5E:C7
add action=mark-connection chain=prerouting comment="Laptop IN Office" \
    disabled=no new-connection-mark=Sonicwall passthrough=yes src-address=\
    192.168.135.199
add action=mark-connection chain=prerouting comment=\
    "For the office Firewall marks connection" disabled=no in-interface=\
    ether1 new-connection-mark=Sonicwall passthrough=yes src-mac-address=\
    00:06:B1:2F:A6:C1
add action=mark-connection chain=prerouting comment=\
    "Locks Mill HLS. Marks Connection" disabled=no in-interface=ether1 \
    new-connection-mark=LocksMill passthrough=yes src-mac-address=\
    00:15:6D:E6:A8:6D
add action=mark-connection chain=prerouting comment="Wes Scheulen" disabled=\
    no in-interface=ether1 new-connection-mark=SSL passthrough=yes \
    src-mac-address=00:15:6D:FA:5D:BE
add action=mark-connection chain=prerouting comment=http://my.agristar.net/ \
    disabled=no dst-address=208.81.96.1 new-connection-mark=Sonicwall \
    passthrough=yes
add action=mark-routing chain=prerouting comment=\
    "Moves connection to PPPoE_2" connection-mark=Sonicwall disabled=no \
    in-interface=ether1 new-routing-mark=to_wan2 passthrough=yes
add action=mark-routing chain=prerouting comment=\
    "Moves connection to PPPoE_3" connection-mark=LocksMill disabled=no \
    in-interface=ether1 new-routing-mark=to_wan3 passthrough=yes
add action=mark-routing chain=prerouting comment=\
    "Moves connection to PPPoE_4" connection-mark=SSL disabled=no \
    in-interface=ether1 new-routing-mark=to_wan4 passthrough=yes
add action=mark-routing chain=prerouting comment=Test2 connection-mark=SSL2 \
    disabled=no in-interface=ether9 new-routing-mark=to_wan3 passthrough=yes

Queue

/queue tree
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 \
    max-limit=20M name=Download-All parent=ether1 priority=8
/queue type
set default kind=pfifo name=default pfifo-limit=50
set ethernet-default kind=pfifo name=ethernet-default pfifo-limit=50
set wireless-default kind=sfq name=wireless-default sfq-allot=1514 \
    sfq-perturb=5
set synchronous-default kind=red name=synchronous-default red-avg-packet=1000 \
    red-burst=20 red-limit=60 red-max-threshold=50 red-min-threshold=10
set hotspot-default kind=sfq name=hotspot-default sfq-allot=1514 sfq-perturb=\
    5
add kind=pcq name=PCQ_Down pcq-classifier=dst-address pcq-limit=50 pcq-rate=\
    768000 pcq-total-limit=2000
set default-small kind=pfifo name=default-small pfifo-limit=10
/queue simple
add burst-limit=256k/512k burst-threshold=128k/256k burst-time=10s/10s \
    comment="Laren Haslag" direction=both disabled=no dst-address=0.0.0.0/0 \
    interface=all limit-at=0/0 max-limit=128k/256k name="Laren Haslag" \
    parent=none priority=8 queue=default-small/default-small \
    target-addresses=192.168.135.112/32 total-queue=default-small
add burst-limit=0/0 burst-threshold=0/0 burst-time=0s/0s comment=\
    "TerryBelle Wolfe" direction=both disabled=no dst-address=0.0.0.0/0 \
    interface=all limit-at=0/0 max-limit=128k/512k name="TerrieBelle Wolfe" \
    parent=none priority=8 queue=default-small/default-small \
    target-addresses=192.168.135.30/32 total-queue=default-small
add burst-limit=512k/1M burst-threshold=0/0 burst-time=10s/10s comment="" \
    direction=both disabled=no dst-address=0.0.0.0/0 interface=all limit-at=\
    0/0 max-limit=256k/768k name=queue4 parent=none priority=8 queue=\
    default/default target-addresses=192.168.135.168/32 total-queue=\
    default-small
add burst-limit=256k/768k burst-threshold=128k/512k burst-time=10s/10s \
    comment="Freedom Products" direction=both disabled=no dst-address=\
    0.0.0.0/0 interface=all limit-at=0/0 max-limit=128k/512k name=\
    "Freedom Products" parent=none priority=8 queue=\
    default-small/default-small target-addresses=192.168.135.33/32 \
    total-queue=default-small
add burst-limit=0/0 burst-threshold=0/0 burst-time=0s/0s comment="Solid Rock" \
    direction=both disabled=no dst-address=0.0.0.0/0 interface=all limit-at=\
    0/0 max-limit=256k/768k name="Solid Rock" parent=none priority=8 queue=\
    default-small/default-small target-addresses=192.168.135.50/32 \
    total-queue=default-small
add burst-limit=0/0 burst-threshold=0/0 burst-time=0s/0s comment="" \
    direction=both disabled=no dst-address=0.0.0.0/0 interface=all limit-at=\
    0/0 max-limit=128k/384k name="Cathay Branson" parent=none priority=8 \
    queue=default-small/default-small target-addresses=192.168.135.68/32 \
    total-queue=default-small
/queue tree
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=768k \
    max-limit=1M name="User 768d" packet-mark=768d parent=Download-All \
    priority=2 queue=PCQ_Down
/queue interface
set ether1 queue=ethernet-default
set ether2 queue=ethernet-default
set ether3 queue=ethernet-default
set ether4 queue=ethernet-default
set ether5 queue=ethernet-default
set ether6 queue=ethernet-default
set ether7 queue=ethernet-default
set ether8 queue=ethernet-default
set ether9 queue=ethernet-default
set pppoe-out1 queue=default
set pppoe-out2 queue=default
set pppoe-out3 queue=default
set pppoe-out4 queue=default
2

Place the chain on forward instead of prerouting and set up another rule that will mark packets with a dst-address of the same IP.

The way you currently have it set up is it will only mark the upload traffic and not download. By marking packets directly you only get one direction of the connection, not the entire thing. You need them in the forward chain because then the the dst-nat will have happened so the router knows where the packet is headed (your server) and it’s before scr-nat happens so it knows where an upload packet has come from.
http://wiki.mikrotik.com/wiki/Packet_Flow#Diagram

3

ok, I’ve done that and now it appears my downloads are getting limited like i want, however, when i tried to do the upload, i ran in to a problem, it looks like the packets are getting marked, but they are not getting to the queue..

Mangle

/ip firewall mangle
add action=mark-packet chain=forward comment="User 768 Download" disabled=no \
    dst-address-list=768d new-packet-mark=768d passthrough=no
add action=mark-packet chain=forward comment="User 768 Upload marks" \
    disabled=no new-packet-mark=256u passthrough=no src-address-list=768d

Queue

/queue tree
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 \
    max-limit=20M name=Download-All parent=ether1 priority=8
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 \
    max-limit=3M name=Upload-All parent=ether1 priority=8
/queue type
set default kind=pfifo name=default pfifo-limit=50
add kind=pcq name=PCQ_Down pcq-classifier=dst-address pcq-limit=50 pcq-rate=\
    768000 pcq-total-limit=2000
add kind=pcq name=PCQ_Up pcq-classifier=src-address pcq-limit=50 pcq-rate=\
    256000 pcq-total-limit=2000
/queue tree
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=768k \
    max-limit=1M name="User 768d" packet-mark=768d parent=Download-All \
    priority=5 queue=PCQ_Down
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=256k \
    max-limit=3M name=User256up packet-mark=256u parent=Upload-All priority=5 \
    queue=PCQ_Up

*******edit
fixed it. changed the parent of the upload to global out and its working now.

4

Your parents for Download-All and Upload-All are wrong. The upload parent should be your WAN interface, and your download should be you WAN. The router can’t do anything about the traffic it is receiving on an interface, so the queue does no good there. It can only control what it is sending out of an interface.

5

I have 4 “wan” interfaces per my first post. (4 dsl lines shared using PCC) I only posted the changes i made. it seems to work with the download parent as ether1 (that’s my LAN interface) and the upload parent as global-out. is it not supposed to work like that?

6

Look at the packet flow diagram I linked, it helps to understand what is going on inside the router and where you need to place things. If it’s working on Global-Out for you that’s good. It just personally confuses me when using the Globals so I much prefer to place the queues directly on the interfaces. The Global-x will also affect all upload interfaces exactly the same and you need to be sure your max limit on that interface includes all upload capabilities of all 4 DSL lines. If you want finer control over what each interface is going to do, then it makes more sense to place a queue on each upload interface.