queue broke after upgrade

hwmonkey · February 4, 2016, 2:48am

After an upgrade (past 6 months), my queues have been malfunctioning. The intent (which used to work) was:

Create a parent queue which is set below total throughput so that packet prioritization can occur
Establish IP based queues for non-LAN activity (using sets of IP addresses).

Queue tree is working correctly. The simple queue target upload/download used to work perfectly without using Total Max Limits. When it broke, I added Max Limits. The max limits work, but the target upload and download do not work. As a result, an individual user can achieve the upload limits of the network.

HELP, please.
System is RB2011UAS-2HnD running 6.33.5

/queue type
set 0 pfifo-limit=25
set 9 pfifo-limit=8
/queue simple
add comment="Limit all BW on bridge2 to 15M" max-limit=14500k/1M name=B2-Parent queue=default/default target=bridge2_Uplink \
    total-max-limit=15M total-queue=default
add dst=bridge2_Uplink name=VoIP priority=2/2 queue=default/default target=192.168.3.200/32,192.168.3.204/32 total-queue=default
add burst-time=30s/30s dst=bridge2_Uplink name=Bridge parent=B2-Parent queue=default/default target=192.168.3.203/32 total-queue=\
    default
add burst-limit=5M/650k burst-threshold=4500k/550k burst-time=3s/3s dst=bridge2_Uplink limit-at=4M/500k max-limit=4M/500k name=\
    User1 parent=B2-Parent queue=default/default target="192.168.3.50/32,192.168.3.51/32,192.168.3.52/32,192.168.3.53/32,192.168.3.54/32,\
    192.168.3.55/32,192.168.3.56/32,192.168.3.57/32,192.168.3.58/32,192.168.3.59/32" total-max-limit=5M total-queue=default
add burst-limit=5M/650k burst-threshold=4500k/550k burst-time=30s/30s max-limit=4M/500k name="18 All Others" parent=B2-Parent queue=\
    default/default target=192.168.3.0/24 total-max-limit=5M total-queue=default
/queue tree
add limit-at=1M max-limit=1M name="2 TOP_PRIORITY_UP" packet-mark=VOIP parent=bridge2_Uplink queue=default
add limit-at=1M max-limit=1M name="3 NORM_PRIORITY_UP" parent=bridge2_Uplink queue=default
add limit-at=10M max-limit=10M name="2 TOP_PRIORITY_DOWN" packet-mark=VOIP parent=bridge1_LAN queue=default
add burst-limit=16M burst-threshold=15M burst-time=5s limit-at=14M max-limit=16M name="3 NORM_PRIORITY_DOWN" parent=bridge1_LAN \
    queue=default
add max-limit=999M name="1 LOCAL" parent=global queue=default
add name=LAN packet-mark=LAN parent="1 LOCAL" queue=default
add name=VOIP_U packet-mark=VOIP parent="2 TOP_PRIORITY_UP" priority=2 queue=default
add name=VOIP_D packet-mark=VOIP parent="2 TOP_PRIORITY_DOWN" priority=2 queue=default
add name=ACK_U packet-mark=ACK parent="3 NORM_PRIORITY_UP" priority=3 queue=default
add name=ACK_D packet-mark=ACK parent="3 NORM_PRIORITY_DOWN" priority=3 queue=default
add name=DNS_U packet-mark=DNS parent="3 NORM_PRIORITY_UP" priority=4 queue=default
add name=DNS_D packet-mark=DNS parent="3 NORM_PRIORITY_DOWN" priority=4 queue=default
add name=HTTP_U packet-mark=HTTP parent="3 NORM_PRIORITY_UP" priority=5 queue=default
add name=HTTP_D packet-mark=HTTP parent="3 NORM_PRIORITY_DOWN" priority=5 queue=default
add name=HTTP_BIG_U packet-mark=HTTP_BIG parent="3 NORM_PRIORITY_UP" priority=6 queue=default
add name=HTTP_BIG_D packet-mark=HTTP_BIG parent="3 NORM_PRIORITY_DOWN" priority=6 queue=default
add name=OTHER_U packet-mark=OTHER parent="3 NORM_PRIORITY_UP" priority=7 queue=default
add name=OTHER_D packet-mark=OTHER parent="3 NORM_PRIORITY_DOWN" priority=7 queue=default
/queue interface
set eth1 queue=default
/ip firewall mangle
add action=add-src-to-address-list address-list=Infected address-list-timeout=1h chain=prerouting connection-state=new disabled=yes \
    dst-port=445 limit=5,10 protocol=tcp
add action=mark-packet chain=forward dst-address=192.168.3.0/24 new-packet-mark=LAN src-address=192.168.3.0/24
add action=mark-packet chain=forward comment=Ooma new-packet-mark=VOIP passthrough=no src-address=192.168.3.200
add action=mark-packet chain=forward comment=Ooma dst-address=192.168.3.200 new-packet-mark=VOIP passthrough=no
add action=mark-packet chain=forward comment=MagicJack new-packet-mark=VOIP passthrough=no src-address=192.168.3.204
add action=mark-packet chain=forward comment=MagicJack dst-address=192.168.3.204 new-packet-mark=VOIP passthrough=no
add action=mark-connection chain=prerouting comment=DNS connection-state=new new-connection-mark=DNS port=53 protocol=udp
add action=mark-packet chain=prerouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=postrouting connection-state=new new-connection-mark=DNS port=53 protocol=udp
add action=mark-packet chain=postrouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-packet chain=postrouting comment=ACK new-packet-mark=ACK packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=prerouting new-packet-mark=ACK packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-connection chain=prerouting comment=HTTP connection-mark=!HTTP_BIG connection-state=new new-connection-mark=HTTP \
    port=80,443 protocol=tcp
add action=mark-connection chain=prerouting connection-bytes=500000-0 connection-mark=HTTP connection-rate=200k-100M \
    new-connection-mark=HTTP_BIG protocol=tcp
add action=mark-packet chain=prerouting connection-mark=HTTP_BIG new-packet-mark=HTTP_BIG passthrough=no
add action=mark-packet chain=prerouting connection-mark=HTTP new-packet-mark=HTTP passthrough=no
add action=mark-connection chain=prerouting comment=OTHER connection-mark=no-mark new-connection-mark=OTHER
add action=mark-packet chain=prerouting connection-mark=OTHER new-packet-mark=OTHER passthrough=no

jarda · February 4, 2016, 4:21am

Make sure that the fasttrack is disabled. When packets go by fasttrack they are not catched by queues.

hwmonkey · February 4, 2016, 12:15pm

Thanks. It is disabled.
ip firewall and fastpath are disabled on the bridge. Enabling the ip firewall causes all queuing to be bypassed.

jarda · February 4, 2016, 12:18pm

Just firewall doesn’t do it. Check what its rules do…

hwmonkey · February 5, 2016, 2:13am

I will restate: enabling firewall on the bridge results in bypass of queues. The firewall itself is functioning normally.

# feb/04/2016 21:08:04 by RouterOS 6.33.5
# software id = 7BM8-S335
#
/ip firewall address-list
add address=192.168.3.0/24 list=allow
/ip firewall connection tracking
set enabled=yes generic-timeout=5m tcp-established-timeout=5m
/ip firewall filter
add action=drop chain=forward comment="default configuration" \
    connection-state=invalid
add action=log chain=input comment="Log it" disabled=yes log-prefix=\
    "DROP INPUT" src-address=!192.168.3.0/24
add action=drop chain=input comment="Guest / MyArea Wall" dst-address=\
    192.168.3.0/24 src-address=192.168.51.0/24
add action=drop chain=forward dst-address=192.168.51.0/24 src-address=\
    192.168.3.0/24
add action=drop chain=forward disabled=yes src-address=192.168.111.0/24
add action=drop chain=input disabled=yes src-address=192.168.111.0/24
add action=drop chain=input comment="Wii / MyArea Wall" dst-address=\
    192.168.3.0/24 src-address=192.168.1.0/24
add action=drop chain=forward disabled=yes dst-address=192.168.1.0/24 \
    src-address=192.168.3.0/24
add action=drop chain=input disabled=yes dst-port=53 in-interface=eth1 \
    protocol=udp
add chain=input comment="Allow limited pings" limit=12/1h,10 protocol=icmp
add action=drop chain=input comment="Drop excess pings" disabled=yes \
    protocol=icmp
add action=add-src-to-address-list address-list="port scanners" \
    address-list-timeout=2w chain=input comment="Port scanners to list " \
    protocol=tcp psd=21,3s,3,1
add action=add-src-to-address-list address-list="port scanners" \
    address-list-timeout=2w chain=input comment="NMAP FIN Stealth scan" \
    protocol=tcp tcp-flags=fin,!syn,!rst,!psh,!ack,!urg
add action=add-src-to-address-list address-list="port scanners" \
    address-list-timeout=2w chain=input comment="SYN/FIN scan" protocol=tcp \
    tcp-flags=fin,syn
add action=add-src-to-address-list address-list="port scanners" \
    address-list-timeout=2w chain=input comment="SYN/RST scan" protocol=tcp \
    tcp-flags=syn,rst
add action=add-src-to-address-list address-list="port scanners" \
    address-list-timeout=2w chain=input comment="FIN/PSH/URG scan" protocol=\
    tcp tcp-flags=fin,psh,urg,!syn,!rst,!ack
add action=add-src-to-address-list address-list="port scanners" \
    address-list-timeout=2w chain=input comment="ALL/ALL scan" protocol=tcp \
    tcp-flags=fin,syn,rst,psh,ack,urg
add action=add-src-to-address-list address-list="port scanners" \
    address-list-timeout=2w chain=input comment="NMAP NULL scan" protocol=tcp \
    tcp-flags=!fin,!syn,!rst,!psh,!ack,!urg
add action=drop chain=input comment="dropping port scanners" \
    src-address-list="port scanners"
add action=drop chain=input comment="Block NTP DDoS" connection-state=new \
    dst-port=123 protocol=udp src-address=!192.168.3.0/24
add action=log chain=input disabled=yes dst-address=!192.168.3.0/24 protocol=\
    tcp src-address=192.168.3.0/24
add action=drop chain=input dst-address=192.168.3.255 dst-port=137 protocol=\
    udp
add action=drop chain=input dst-port=137 protocol=udp src-address=\
    !192.168.3.0/24
add chain=input comment="Accept established connections" connection-state=\
    established
add chain=input comment="Accept related connections" connection-state=related
add chain=input comment="Ooma -anything goes" src-address=192.168.3.200
add action=drop chain=input comment="Drop invalid connections" \
    connection-state=invalid
add action=drop chain=forward dst-port=445 protocol=tcp src-address-list=\
    Infected
add action=drop chain=forward dst-port=445 protocol=tcp src-address-list=\
    Infected
add chain=input comment=l2tp dst-port=8291 protocol=udp
add chain=input comment="Encapsulating Security Payload for IPv4 (ESP)" \
    protocol=ipsec-esp
add chain=input comment="SSH for secure shell" dst-port=22 protocol=tcp \
    src-address-list=allow
add chain=input comment="NAT traversal" dst-port=4500 protocol=udp
add chain=input comment=VPN dst-port=500 protocol=udp src-port=500
add chain=input comment=winbox dst-port=1701 protocol=tcp
add chain=input comment=VPN dst-port=1701 protocol=udp src-port=1701
add chain=input comment=winbox dst-port=8291 protocol=tcp
add action=drop chain=input dst-port=53 in-interface=bridge2_Uplink protocol=\
    udp
add action=drop chain=input comment="Block mikrotik discovery" dst-port=5678 \
    in-interface=bridge2_Uplink protocol=udp
add chain=input comment="Local LAN" dst-address=192.168.3.0/24 src-address=\
    192.168.3.0/24
add chain=input disabled=yes dst-address=192.168.1.0/24 in-interface=\
    wlan2_wii src-address=192.168.1.0/24
add chain=input dst-address=192.168.51.0/24 src-address=192.168.51.0/24
add chain=input comment="Local LAN" dst-address=239.0.0.0/8 src-address=\
    192.168.3.0/24
add chain=input comment="Local LAN" dst-address=255.255.255.0/24 src-address=\
    192.168.3.0/24
add chain=input comment="Local LAN" dst-address=255.255.255.255 src-address=\
    0.0.0.0
add action=drop chain=input dst-address=255.255.255.255 in-interface=\
    bridge2_Uplink protocol=udp
add action=log chain=input comment="Log everything else" disabled=yes \
    log-prefix="DROP INPUT"
add action=drop chain=input comment="Drop everything else"
/ip firewall mangle
add action=add-src-to-address-list address-list=Infected \
    address-list-timeout=1h chain=prerouting connection-state=new disabled=\
    yes dst-port=445 limit=5,10 protocol=tcp
add action=mark-packet chain=forward dst-address=192.168.3.0/24 \
    new-packet-mark=LAN src-address=192.168.3.0/24
add action=mark-packet chain=forward comment=Ooma new-packet-mark=VOIP \
    passthrough=no src-address=192.168.3.200
add action=mark-packet chain=forward comment=Ooma dst-address=192.168.3.200 \
    new-packet-mark=VOIP passthrough=no
add action=mark-packet chain=forward comment=MagicJack new-packet-mark=VOIP \
    passthrough=no src-address=192.168.3.204
add action=mark-packet chain=forward comment=MagicJack dst-address=\
    192.168.3.204 new-packet-mark=VOIP passthrough=no
add action=mark-connection chain=prerouting comment=DNS connection-state=new \
    new-connection-mark=DNS port=53 protocol=udp
add action=mark-packet chain=prerouting connection-mark=DNS new-packet-mark=\
    DNS passthrough=no
add action=mark-connection chain=postrouting connection-state=new \
    new-connection-mark=DNS port=53 protocol=udp
add action=mark-packet chain=postrouting connection-mark=DNS new-packet-mark=\
    DNS passthrough=no
add action=mark-packet chain=postrouting comment=ACK new-packet-mark=ACK \
    packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=prerouting new-packet-mark=ACK packet-size=0-123 \
    passthrough=no protocol=tcp tcp-flags=ack
add action=mark-connection chain=prerouting comment=HTTP connection-mark=\
    !HTTP_BIG connection-state=new new-connection-mark=HTTP port=80,443 \
    protocol=tcp
add action=mark-connection chain=prerouting connection-bytes=500000-0 \
    connection-mark=HTTP connection-rate=200k-100M new-connection-mark=\
    HTTP_BIG protocol=tcp
add action=mark-packet chain=prerouting connection-mark=HTTP_BIG \
    new-packet-mark=HTTP_BIG passthrough=no
add action=mark-packet chain=prerouting connection-mark=HTTP new-packet-mark=\
    HTTP passthrough=no
add action=mark-connection chain=prerouting comment=OTHER connection-mark=\
    no-mark new-connection-mark=OTHER
add action=mark-packet chain=prerouting connection-mark=OTHER \
    new-packet-mark=OTHER passthrough=no
/ip firewall nat
add action=dst-nat chain=dstnat dst-address=192.168.111.0/24 dst-port=10222 \
    protocol=tcp to-addresses=192.168.3.101 to-ports=22
add action=dst-nat chain=dstnat comment=Fileshare disabled=yes dst-address=\
    192.168.111.0/24 dst-port=6881 protocol=udp to-addresses=192.168.3.101 \
    to-ports=6881
add action=dst-nat chain=dstnat dst-address=192.168.111.0/24 dst-port=23389 \
    protocol=tcp to-addresses=192.168.3.102 to-ports=3389
add action=dst-nat chain=dstnat dst-address=192.168.111.0/24 dst-port=13389 \
    protocol=tcp to-addresses=192.168.3.101 to-ports=3389
add action=dst-nat chain=dstnat dst-address=192.168.3.0/24 dst-port=13389 \
    protocol=tcp to-addresses=192.168.3.101 to-ports=3389
add action=dst-nat chain=dstnat dst-address=192.168.111.0/24 dst-port=20380 \
    protocol=tcp to-addresses=192.168.3.203 to-ports=80
add action=dst-nat chain=dstnat dst-address=192.168.111.0/24 dst-port=20080 \
    protocol=tcp to-addresses=192.168.3.200 to-ports=80
add action=masquerade chain=srcnat out-interface=bridge2_Uplink to-addresses=\
    0.0.0.0

jarda · February 5, 2016, 5:26am

Ok. Looks you are not using fasttrack. Is the same problem with 6.32.3 too? I am keeping this version to prevent useless problems. If not, make the supout file and send it to the support.

macgaiver · February 5, 2016, 5:40am

hwmonkey:

/queue simple
add comment="Limit all BW on bridge2 to 15M" max-limit=14500k/1M [b]name=B2-Parent[/b] queue=default/default[b] target=bridge2_Uplink[/b] \
    total-max-limit=15M total-queue=default
add [b]dst=bridge2_Uplink[/b] name=VoIP priority=2/2 queue=default/default target=192.168.3.200/32,192.168.3.204/32 total-queue=default
add burst-time=30s/30s dst=bridge2_Uplink name=Bridge [b]parent=B2-Parent[/b] queue=default/default target=192.168.3.203/32 total-queue=\
    default
add burst-limit=5M/650k burst-threshold=4500k/550k burst-time=3s/3s dst=bridge2_Uplink limit-at=4M/500k max-limit=4M/500k name=\
    User1 parent=B2-Parent queue=default/default target="192.168.3.50/32,192.168.3.51/32,192.168.3.52/32,192.168.3.53/32,192.168.3.54/32,\
    192.168.3.55/32,192.168.3.56/32,192.168.3.57/32,192.168.3.58/32,192.168.3.59/32" total-max-limit=5M total-queue=default
add burst-limit=5M/650k burst-threshold=4500k/550k burst-time=30s/30s max-limit=4M/500k name="18 All Others" parent=B2-Parent queue=\
    default/default target=192.168.3.0/24 total-max-limit=5M total-queue=default

Your first child queue is pointing the opposite direction to your parent queue, no wonder it doesn’t work. I have question - how did this possiblly worked before??

Golden rules when making sinple queue structure.

Choose one direction for all queues - usually target is client IP or interfaces where client is connected
in simple queue, parents capture only traffic that is captured by its children, so you need to make sure that child queues capture all the traffic, maybe even create copy of parent queue as last child.

hwmonkey · February 5, 2016, 12:56pm

Your first child queue is pointing the opposite direction to your parent queue, no wonder it doesn’t work. I have question - how did this possiblly worked before??

The first queue is for VOIP and is intended to omit this VOIP traffic (and only this VOIP traffic) from all queuing. The other children are the problem children. They are limiting related to total traffic and not by up/down. Does that help clarify how it was working?

hwmonkey · March 19, 2016, 2:18pm

I made zero changes.

After an upgrade to 6.34.3, queues are working again.