QoS - video stream buffering

RouterOS General
1

Hey folks,

I’ve read a lot of posts on how to limit streaming and very few on how to improve it. I have enough bandwidth to support 1-2 streams for my clients, however they are buffering like crazy. I have 100Mbit circuit, average load at night is 50Mbit. i have folks on 10Mbit down plans that cant do one netflix stream during these busy times. This sucks cause i keep getting complaints.

I tried implementing this http://gregsowell.com/?p=4665, and upped the limits by 10 (his example was 10M and i have 100M), but it doesn’t seem to be working.

Here’s what i got. yeah i suck at QoS, but I’m going to be burned at the stake if i don’t figure this out.

/ip firewall layer7-protocol
add comment=“” name=speedtest-servers regexp=“^.(get|GET).+speedtest.$”
add comment=“” name=torrent-wwws regexp=“^.(get|GET).+(torrent|thepiratebay|i
sohunt|entertane|demonoid|btjunkie|mininova|flixflux|vertor|h33t|zoozle|bi
tnova|bitsoup|meganova|fulldls|btbot|fenopy|gpirate|commonbits).$”
add comment=“” name=torrent-dns regexp=“^.+(torrent|thepiratebay|isohunt|enter
tane|demonoid|btjunkie|mininova|flixflux|vertor|h33t|zoozle|bitnova|bitsou
p|meganova|fulldls|btbot|fenopy|gpirate|commonbits).$"
add comment=“” name=netflix regexp="^.(get|GET).+(netflix).$"
add comment=“” name=mp4 regexp="^.(get|GET).+\.mp4.$"
add comment=“” name=swf regexp="^.(get|GET).+\.swf.$"
add comment=“” name=flv regexp="^.(get|GET).+\.flv.$"
add name=video regexp="^.(get|GET).+(\.flv|\.mp4|netflix|\.swf).*$”

/ip firewall address-list
add address=10.0.1.0/24 comment=“” disabled=no list=internal-nets
add address=10.0.2.0/24 comment=“” disabled=no list=internal-nets
add address=/30 comment=“” disabled=no list=external-nets
add address=/24 comment=“” disabled=no list=external-nets

/ip firewall mangle
add action=mark-packet chain=prerouting comment=“internal-traffic packet mark” dst-address-list=
internal-nets new-packet-mark=internal-traffic passthrough=no src-address-list=internal-nets
add action=mark-packet chain=prerouting comment=“customer-servers-in packet mark” dst-address-list=
customer-servers new-packet-mark=customer-servers-in passthrough=no
add action=mark-packet chain=prerouting comment=“admin-in packet mark DNS” in-interface=WAN_XO new-packet-mark=admin-in passthrough=no protocol=udp src-port=53
add action=mark-packet chain=prerouting comment=“admin-in packet mark snmp” dst-port=161
in-interface=WAN_XO new-packet-mark=admin-in passthrough=no protocol=udp
add action=mark-connection chain=prerouting comment=“Remote Protocols admin connection mark”
new-connection-mark=admin port=20,21,22,23,3389,8291 protocol=tcp
add action=mark-connection chain=prerouting comment=“icmp connection mark as admin”
new-connection-mark=admin protocol=icmp src-address-list=internal-nets
add action=mark-packet chain=prerouting comment=“admin-in packet mark” connection-mark=admin
in-interface=WAN_XO new-packet-mark=admin-in passthrough=no
add action=mark-packet chain=prerouting comment=“admin-out packet mark” connection-mark=admin
new-packet-mark=admin-out passthrough=no
add action=mark-connection chain=prerouting comment=“streaming video connection mark” dst-port=80
layer7-protocol=video new-connection-mark=streaming-video protocol=tcp src-address-list=
internal-nets
add action=mark-packet chain=prerouting comment=“streaming video in packet mark” connection-mark=
streaming-video in-interface=WAN_XO new-packet-mark=streaming-video-in passthrough=no
add action=mark-packet chain=prerouting comment=“streaming video out packet mark” connection-mark=
streaming-video new-packet-mark=streaming-video-out passthrough=no
add action=mark-connection chain=prerouting comment=“http traffic connection mark” dst-port=80,443
new-connection-mark=http protocol=tcp src-address-list=internal-nets
add action=mark-connection chain=prerouting comment=“http traffic connection mark”
connection-bytes=5000000-4294967295 dst-port=80,443 new-connection-mark=http-download protocol=
tcp src-address-list=internal-nets
add action=mark-packet chain=prerouting comment=“http in packet mark” connection-mark=http
in-interface=WAN_XO new-packet-mark=http-in passthrough=no
add action=mark-packet chain=prerouting comment=“http out packet mark” connection-mark=http
new-packet-mark=http-out passthrough=no
add action=mark-connection chain=prerouting comment=“wow connetion mark as gaming” dst-port=
1119,3724,6112-6114,4000,6881-6999 new-connection-mark=games protocol=tcp src-address-list=
internal-nets
add action=mark-connection chain=prerouting comment=“eve online connetion mark as gaming”
dst-address=87.237.38.200 new-connection-mark=games src-address-list=internal-nets
add action=mark-connection chain=prerouting comment=“starcraft 2 connetion mark as gaming”
dst-port=1119 new-connection-mark=games protocol=tcp src-address-list=internal-nets
add action=mark-connection chain=prerouting comment=“heros of newerth connetion mark as gaming”
dst-port=11031,11235-11335 new-connection-mark=games protocol=tcp src-address-list=
internal-nets
add action=mark-connection chain=prerouting comment=“steam connetion mark as gaming” dst-port=
27014-27050 new-connection-mark=games protocol=tcp src-address-list=internal-nets
add action=mark-connection chain=prerouting comment=“xbox live connetion mark as gaming” dst-port=
3074 new-connection-mark=games protocol=tcp src-address-list=internal-nets
add action=mark-connection chain=prerouting comment=“ps3 online connetion mark as gaming” dst-port=
5223 new-connection-mark=games protocol=tcp src-address-list=internal-nets
add action=mark-connection chain=prerouting comment=“wii online connetion mark as gaming” dst-port=
28910,29900,29901,29920 new-connection-mark=games protocol=tcp src-address-list=internal-nets
add action=mark-packet chain=prerouting comment=“games packet mark forever-saken-game”
dst-address-list=external-nets new-packet-mark=games-in passthrough=no src-address-list=
forever-saken-game
add action=mark-packet chain=prerouting comment=“games packet mark wow” dst-address-list=
external-nets new-packet-mark=games-in passthrough=no protocol=udp src-port=53,3724
add action=mark-packet chain=prerouting comment=“games packet mark starcraft2” dst-address-list=
external-nets new-packet-mark=games-in passthrough=no protocol=udp src-port=1119,6113
add action=mark-packet chain=prerouting comment=“games packet mark HoN” dst-address-list=
external-nets new-packet-mark=games-in passthrough=no protocol=udp src-port=11031,11235-11335
add action=mark-packet chain=prerouting comment=“games packet mark steam in” dst-address-list=
external-nets new-packet-mark=games-in passthrough=no port=4380,28960,27000-27030 protocol=udp
add action=mark-packet chain=prerouting comment=“games packet mark steam out” dst-port=
53,1500,3005,3101,3478,4379-4380,4380,28960,27000-27030,28960 new-packet-mark=games-out
passthrough=no protocol=udp src-address-list=internal-nets
add action=mark-packet chain=prerouting comment=“games packet mark xbox live” dst-address-list=
external-nets new-packet-mark=games-in passthrough=no protocol=udp src-port=88,3074,3544,4500
add action=mark-packet chain=prerouting comment=“games packet mark ps3 online” dst-address-list=
external-nets new-packet-mark=games-in passthrough=no protocol=udp src-port=3478,3479,3658
add action=mark-packet chain=prerouting comment=“games packet mark in” connection-mark=games
dst-address-list=external-nets new-packet-mark=games-in passthrough=no
add action=mark-packet chain=prerouting comment=“games packet mark out” connection-mark=games
new-packet-mark=games-out passthrough=no
add action=mark-packet chain=prerouting comment=“voip-in packet mark teamspeak” dst-address-list=
external-nets new-packet-mark=voip-in passthrough=no protocol=udp src-port=9987
add action=mark-packet chain=prerouting comment=“voip-out packet mark teamspeak” dst-port=9987
new-packet-mark=voip-out passthrough=no protocol=udp src-address-list=internal-nets
add action=mark-packet chain=prerouting comment=“voip-out packet mark teamspeak” dst-address-list=
external-nets new-packet-mark=voip-in passthrough=no protocol=udp src-port=9987
add action=mark-packet chain=prerouting comment=“voip-in packet mark ventrilo” dst-address-list=
external-nets new-packet-mark=voip-in passthrough=no protocol=udp src-port=3784
add action=mark-packet chain=prerouting comment=“voip-out packet mark ventrilo” dst-port=3784
new-packet-mark=voip-out passthrough=no protocol=udp src-address-list=internal-nets
add action=mark-packet chain=prerouting comment=“voip-in packet mark ventrilo” dst-address-list=
external-nets new-packet-mark=voip-in passthrough=no protocol=tcp src-port=3784
add action=mark-packet chain=prerouting comment=“voip-out packet mark ventrilo” dst-port=3784
new-packet-mark=voip-out passthrough=no protocol=tcp src-address-list=internal-nets
add action=mark-packet chain=prerouting comment=“voip-in packet mark SIP” dst-address-list=
internal-nets new-packet-mark=voip-in passthrough=no port=5060 protocol=tcp
add action=mark-packet chain=prerouting comment=“voip-out packet mark SIP” new-packet-mark=voip-out
passthrough=no port=5060 protocol=tcp src-address-list=internal-nets
add action=mark-packet chain=prerouting comment=“voip-in packet mark udp SIP” dst-address-list=
internal-nets new-packet-mark=voip-in passthrough=no port=5004,5060 protocol=udp
add action=mark-packet chain=prerouting comment=“voip-out packet mark udp SIP” new-packet-mark=
voip-out passthrough=no port=5004,5060 protocol=udp src-address-list=internal-nets
add action=mark-packet chain=prerouting comment=“voip-in packet mark RTP” dst-address-list=
internal-nets new-packet-mark=voip-in packet-size=100-400 passthrough=no port=16348-32768
protocol=udp
add action=mark-packet chain=prerouting comment=“voip-out packet mark RTP” new-packet-mark=voip-in
packet-size=100-400 passthrough=no port=16348-32768 protocol=udp src-address-list=internal-nets
add action=mark-packet chain=prerouting comment=“vpn-in packet mark GRE” in-interface=WAN_XO
new-packet-mark=vpn-in passthrough=no protocol=gre
add action=mark-packet chain=prerouting comment=“vpn-out packet mark GRE” new-packet-mark=vpn-out
passthrough=no protocol=gre
add action=mark-packet chain=prerouting comment=“vpn-in packet mark ESP” in-interface=WAN_XO
new-packet-mark=vpn-in passthrough=no protocol=ipsec-esp
add action=mark-packet chain=prerouting comment=“vpn-out packet mark ESP” new-packet-mark=vpn-out
passthrough=no protocol=ipsec-esp
add action=mark-packet chain=prerouting comment=“vpn-in packet mark VPN UDP ports” in-interface=
WAN_XO new-packet-mark=vpn-in passthrough=no protocol=udp src-port=500,1701,4500
add action=mark-packet chain=prerouting comment=“vpn-out packet mark VPN UDP ports”
new-packet-mark=vpn-out passthrough=no protocol=udp src-port=500,1701,4500
add action=mark-packet chain=prerouting comment=“vpn-in packet mark PPTP” in-interface=WAN_XO
new-packet-mark=vpn-in passthrough=no protocol=tcp src-port=1723
add action=mark-packet chain=prerouting comment=“vpn-out packet mark PPTP” new-packet-mark=vpn-out
passthrough=no protocol=tcp src-port=1723
add action=mark-packet chain=prerouting comment=“all in” in-interface=WAN_XO new-packet-mark=in
passthrough=no
add action=mark-packet chain=prerouting comment=“all out” new-packet-mark=out passthrough=no

/queue type
add kind=pfifo name=streaming-video-in pfifo-limit=1000
add kind=pcq name=games-in-pcq pcq-burst-rate=0 pcq-burst-threshold=0 pcq-burst-time=10s pcq-classifier=dst-address pcq-dst-address-mask=32 pcq-dst-address6-mask=64
pcq-limit=500 pcq-rate=1000k pcq-src-address-mask=32 pcq-src-address6-mask=64 pcq-total-limit=7500000
/queue tree
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0
max-limit=100M name=in parent=global priority=8
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0
max-limit=100M name=out parent=global priority=8
/queue tree
add max-limit=100M name=in parent=global queue=default
add max-limit=100M name=out parent=global queue=default
add limit-at=30M max-limit=100M name=http-in packet-mark=http-in parent=in priority=4 queue=default
add limit-at=40M max-limit=100M name=streaming-video-in packet-mark=streaming-video-in parent=in
priority=3 queue=streaming-video-in
add limit-at=5000k max-limit=100M name=gaming-in packet-mark=games-in parent=in priority=2 queue=
games-in-pcq
add max-limit=100M name=download-in packet-mark=in parent=in queue=default
add max-limit=100M name=upload-out packet-mark=out parent=out queue=default
add limit-at=5000k max-limit=100M name=gaming-out packet-mark=games-out parent=out priority=2 queue=
default
add limit-at=30M max-limit=100M name=http-out packet-mark=http-out parent=out priority=4 queue=default
add limit-at=40M max-limit=100M name=streaming-video-out packet-mark=streaming-video-out parent=out
priority=3 queue=default
add limit-at=10M max-limit=100M name=customer-servers-in packet-mark=customer-servers-in parent=in
priority=1 queue=default
add limit-at=10M max-limit=100M name=customer-servers-out packet-mark=customer-servers-out parent=out
priority=1 queue=default
add limit-at=5000k max-limit=100M name=voip-in packet-mark=voip-in parent=in priority=1 queue=default
add limit-at=5000k max-limit=100M name=vpn-in packet-mark=vpn-in parent=in priority=2 queue=default
add limit-at=5000k max-limit=100M name=voip-out packet-mark=voip-out parent=out priority=1 queue=
default
add limit-at=5000k max-limit=100M name=vpn-out packet-mark=vpn-out parent=out priority=2 queue=default
add limit-at=5000k max-limit=100M name=admin-in packet-mark=admin-in parent=in priority=1 queue=default
add limit-at=5000k max-limit=100M name=admin-out packet-mark=admin-out parent=out priority=1 queue=
default

2

Disclaimer: I don’t run a WISP (or work for any other IPS type organization in that matter any more), but do IT work for a bunch of old people and small businesses.

Since you mention Netflix initially, while I’m not good at QoS on Mikrotik myself, I don’t think this script will help. Or other L7 classifiers for that manner, because:
Netflix has started hosting their videos on the nflxvideo.net domain - the match rules need to be changed a bit. Some shows are still on Akamai, but that is extremely rare.
Netflix has also started streaming over HTTPS, so unless you do MITM I don’t see how L7 connection classifiers will work.

More importantly Netflix doesn’t buffer that much - 100Mbps for streams under 5Mpbs and 200Mbps for streams above 5Mbps. If you can’t manage that you should look for interference at the particular part of the day and get it resolved, or you have significantly oversubscribed and either should throttle the customers a bit more equally, or reduce your plans.

There are few things you can do though if you want to get some idea of what to throttle and how:
Most of the Netflix servers hosting the are in their own AS now, for quite some time. If you have BGP running, you can script the prefixes in the announcements into some address lists and throttle based on that. If you don’t have BGP running - you can pull them from a public BGP looking glass. Please don’t abuse a public looking glass with a script.
To validate where the connections are going to:
If you can connect a machine that is Netflix capable to the customer segment (or rather within the same IP space) and open a video on it press ctrl+alt+shift+d in the Netflix player - you should see a bit of debug info what the servers are. Note down the FQDNs, resolve them, take a look what is the allocation they sit in, and add that allocation to the address lists. Most likely they will be within the Netflix AS.
If you can’t connect a machine in the same IP space, there is another way that IS NOT RECOMMENDED and it works only if your customer’s DNS traffic hits your own DNS cache resolvers. For the one in the MTK you can see the cache and look the FQDNs there, and figure it out.

But in general I suggest fixing the network overprovisioning.