Community discussions

MikroTik App
 
User avatar
azurtem
Trainer
Trainer
Topic Author
Posts: 217
Joined: Mon May 16, 2011 5:35 pm
Location: Nice, France
Contact:

Traffic shaping - prioritizing the essentials

Thu Dec 13, 2012 10:52 am

Hi

This past week I have been working on a bandwidth traffic
shaper for one of my hotel clients.

Situated at the end of an ADSL line all they have is 2MB/700K,
so come summer the user complaints begin.

They already have a Zyxel N4100 hotspot in place.

I started off with a 450, which I soon replaced with a 450G
afraid that my initial choice might not be powerful enough
for such a task. What do you think ?

I'm going to install the 450G between the Zyxel (WAN port)
and the Orange Livebox.

There is a wealth of information about queues and a fair
amount about traffic shaping, but in my opinion it is a little
disparate with regards to the latter.

I had initially began mangling everything in mind, essentially
based on the available L7 pattern list. Having realized the
ridicule of such an approach, I came around.

Basically I only mangle the traffic which I want to prioritize :
dns, http, smtp, pop3, imap, ssl. Here is the mangle rule list :
/ip firewall mangle
add action=mark-connection chain=forward disabled=no dst-port=53 \
    new-connection-mark=dns-cnx passthrough=yes protocol=udp
add action=mark-packet chain=forward connection-mark=dns-cnx disabled=no \
    new-packet-mark=dns-in out-interface="ether1 - Zyxel N4100 (HOTSPOT WAN)" \
    passthrough=no
add action=mark-packet chain=forward connection-mark=dns-cnx disabled=no \
    new-packet-mark=dns-out out-interface="ether2 - LiveBox - Internet" \
    passthrough=no
add action=mark-connection chain=forward disabled=yes dst-port=80 \
    new-connection-mark=http-cnx passthrough=yes protocol=tcp
add action=mark-connection chain=forward disabled=no layer7-protocol=http \
    new-connection-mark=http-cnx passthrough=yes
add action=mark-packet chain=forward connection-mark=http-cnx disabled=no \
    new-packet-mark=http-in out-interface=\
    "ether1 - Zyxel N4100 (HOTSPOT WAN)" passthrough=no
add action=mark-packet chain=forward connection-mark=http-cnx disabled=no \
    new-packet-mark=http-out out-interface="ether2 - LiveBox - Internet" \
    passthrough=no
add action=mark-connection chain=forward disabled=no dst-port=25,587 \
    new-connection-mark=smtp-cnx passthrough=yes protocol=tcp
add action=mark-packet chain=forward connection-mark=smtp-cnx disabled=no \
    new-packet-mark=smtp-in out-interface=\
    "ether1 - Zyxel N4100 (HOTSPOT WAN)" passthrough=no
add action=mark-packet chain=forward connection-mark=smtp-cnx disabled=no \
    new-packet-mark=smtp-out out-interface="ether2 - LiveBox - Internet" \
    passthrough=no
add action=mark-connection chain=forward disabled=no dst-port=110 \
    new-connection-mark=pop3-cnx passthrough=yes protocol=tcp
add action=mark-packet chain=forward connection-mark=pop3-cnx disabled=no \
    new-packet-mark=pop3-in out-interface=\
    "ether1 - Zyxel N4100 (HOTSPOT WAN)" passthrough=no
add action=mark-packet chain=forward connection-mark=pop3-cnx disabled=no \
    new-packet-mark=pop3-out out-interface="ether2 - LiveBox - Internet" \
    passthrough=no
add action=mark-connection chain=forward disabled=no dst-port=143 \
    new-connection-mark=imap-cnx passthrough=yes protocol=tcp
add action=mark-packet chain=forward connection-mark=imap-cnx disabled=no \
    new-packet-mark=imap-in out-interface=\
    "ether1 - Zyxel N4100 (HOTSPOT WAN)" passthrough=no
add action=mark-packet chain=forward connection-mark=imap-cnx disabled=no \
    new-packet-mark=imap-out out-interface="ether2 - LiveBox - Internet" \
    passthrough=no
add action=mark-connection chain=forward disabled=no layer7-protocol=ssl \
    new-connection-mark=ssl-cnx passthrough=yes
add action=mark-packet chain=forward connection-mark=ssl-cnx disabled=no \
    new-packet-mark=ssl-in out-interface="ether1 - Zyxel N4100 (HOTSPOT WAN)" \
    passthrough=no
add action=mark-packet chain=forward connection-mark=ssl-cnx disabled=no \
    new-packet-mark=ssl-out out-interface="ether2 - LiveBox - Internet" \
    passthrough=no
add action=mark-packet chain=forward disabled=no new-packet-mark=all-else-in \
    out-interface="ether1 - Zyxel N4100 (HOTSPOT WAN)" passthrough=no
add action=mark-packet chain=forward disabled=no new-packet-mark=all-else-out \
    out-interface="ether2 - LiveBox - Internet" passthrough=no
The all-else rule at the end literally refers to anything not
previously specified. I don't block this other traffic, I mean,
if the bandwidth is free, let them use it.

I read here and there that it isn't necessary to specify both
upload and download mangle rules for each protocol, and
rely on the queue-type classifier, but I don't see how you
could apply the queues in a precise manner.

For HTTP, I preferred a L7 filter rather than the classic tcp/80
hoping that this will weed out online videos and video streaming
and relegate them to the all-else rule.

What follows is the corresponding queue-tree :
/queue tree
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 \
    max-limit=2M name=TOTAL_DOWNLOAD packet-mark="" parent=global-out \
    priority=1
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 \
    max-limit=700k name=TOTAL_UPLOAD packet-mark="" parent=global-out \
    priority=1
/queue type
set 0 kind=pfifo name=default pfifo-limit=50
set 1 kind=pfifo name=ethernet-default pfifo-limit=50
set 2 kind=sfq name=wireless-default sfq-allot=1514 sfq-perturb=5
set 3 kind=red name=synchronous-default red-avg-packet=1000 red-burst=20 \
    red-limit=60 red-max-threshold=50 red-min-threshold=10
set 4 kind=sfq name=hotspot-default sfq-allot=1514 sfq-perturb=5
add kind=pcq name=PCQ_download pcq-burst-rate=0 pcq-burst-threshold=0 \
    pcq-burst-time=10s pcq-classifier=dst-address pcq-dst-address-mask=32 \
    pcq-dst-address6-mask=128 pcq-limit=30 pcq-rate=0 pcq-src-address-mask=32 \
    pcq-src-address6-mask=128 pcq-total-limit=2000
add kind=pcq name=PCQ_upload pcq-burst-rate=0 pcq-burst-threshold=0 \
    pcq-burst-time=10s pcq-classifier=src-address pcq-dst-address-mask=32 \
    pcq-dst-address6-mask=128 pcq-limit=30 pcq-rate=0 pcq-src-address-mask=32 \
    pcq-src-address6-mask=128 pcq-total-limit=2000
set 7 kind=none name=only-hardware-queue
set 8 kind=mq-pfifo mq-pfifo-limit=50 name=multi-queue-ethernet-default
set 9 kind=pfifo name=default-small pfifo-limit=10
/queue tree
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 \
    max-limit=0 name="1: dns-in" packet-mark=dns-in parent=TOTAL_DOWNLOAD \
    priority=1 queue=PCQ_download
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 \
    max-limit=0 name="2: http-in" packet-mark=http-in parent=TOTAL_DOWNLOAD \
    priority=2 queue=PCQ_download
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 \
    max-limit=0 name="3: smtp-in" packet-mark=smtp-in parent=TOTAL_DOWNLOAD \
    priority=3 queue=PCQ_download
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 \
    max-limit=0 name="3: ssl-in" packet-mark=ssl-in parent=TOTAL_DOWNLOAD \
    priority=3 queue=PCQ_download
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 \
    max-limit=0 name="4: pop3-in" packet-mark=pop3-in parent=TOTAL_DOWNLOAD \
    priority=4 queue=PCQ_download
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 \
    max-limit=0 name="5: ssh-in" packet-mark=ssh-in parent=TOTAL_DOWNLOAD \
    priority=5 queue=PCQ_download
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 \
    max-limit=0 name="5: imap-in" packet-mark=imap-in parent=TOTAL_DOWNLOAD \
    priority=5 queue=PCQ_download	
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 \
    max-limit=0 name="6: all-else-in" packet-mark=all-else-in parent=\
    TOTAL_DOWNLOAD priority=6 queue=PCQ_download
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 \
    max-limit=0 name="1: dns_out" packet-mark=dns-out parent=TOTAL_UPLOAD \
    priority=1 queue=PCQ_upload
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 \
    max-limit=0 name="2: http-out" packet-mark=http-out parent=TOTAL_UPLOAD \
    priority=2 queue=PCQ_upload
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 \
    max-limit=0 name="3: smtp-out" packet-mark=smtp-out parent=TOTAL_UPLOAD \
    priority=3 queue=PCQ_upload
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 \
    max-limit=0 name="3: ssl-out" packet-mark=ssl-out parent=TOTAL_UPLOAD \
    priority=3 queue=PCQ_upload
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 \
    max-limit=0 name="4: pop3-out" packet-mark=pop3-out parent=TOTAL_UPLOAD \
    priority=4 queue=PCQ_upload
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 \
    max-limit=0 name="5: imap-out" packet-mark=imap-out parent=TOTAL_UPLOAD \
    priority=5 queue=PCQ_upload
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 \
    max-limit=0 name="5: ssh-out" packet-mark=ssh-out parent=TOTAL_UPLOAD \
    priority=5 queue=PCQ_upload
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 \
    max-limit=0 name="6: all-else-out" packet-mark=all-else-out parent=\
    TOTAL_UPLOAD priority=6 queue=PCQ_upload
During my tests things seem to be shaping up properly,
pun intended.

There was however one instance when HTTP traffic didn't
get through i.e. didn't get the priority it was assigned.

I use Thunderbird as my mail client, in imap mode. Smtp
is set to 465 SSL/TLS. My company account is set to 143
STARTTLS, while my Gmail account is set to 993 SSL/TLS.

My test environment is comprised of a video streaming
of Private Practice on TV-Links, a Youtube viewing of a
sufficiently long lasting Friends episode, while uTorrent
would be downloading the DVD image of the latest Debian
release and I would also have a streaming radio station
playing on Winamp.

At this point the shaping works apparently flawlessly, as
tested on my PC, my iPhone and my wirelessly connected
notebook.

I then send a 9MB email (PDF attachment) with my company
account to my Gmail account from my PC.

The procedure, in my case, with IMAP is that the email is sent,
and then a copy of it is stored in the send folder i.e. the file is
uploaded twice in a row to my ISP's mail server.

For some reason, which bewilders me, during the second part,
the copy to the sent folder, HTTP cannot get through - it is
totally blocked. Nothing else to do but wait. Once the copy is
over, everything returns to normal.

There's probably something wrong in my mangling/queue-tree
logic that hasn't popped up in my line of sight yet. Any ideas ?

thanks for reading

yann

Who is online

Users browsing this forum: Bing [Bot], hla14, Majestic-12 [Bot], Rxafast and 137 guests