Community discussions

MikroTik App
 
User avatar
janisk
MikroTik Support
MikroTik Support
Topic Author
Posts: 6263
Joined: Tue Feb 14, 2006 9:46 am
Location: Riga, Latvia

Burst for each TCP connection

Thu Dec 21, 2006 2:01 pm

This is little how-to create manual burst using queue tree.

As it is bandwidth control using queue tree first we need to mangle traffic

first i mangle all connections, then i mark first 2Mbytes then i mark the rest of packets
/ip firewall mangle add chain=forward protocol=tcp action=mark-connection new-connection-mark=new_conn passthrough=yes comment="mark all new connections" disabled=no 
/ip firewall mangle add chain=forward protocol=tcp  connection-mark=new_conn connection-bytes=0-2000000 action=mark-packet new-packet-mark=new_packet passthrough=no comment="mark packets" disabled=no 
/ip firewall mangle add chain=forward protocol=tcp connection-mark=new_conn action=mark-packet new-packet-mark=old_packets passthrough=no comment="marking old packets" disabled=no 
now, when packets has been marked i need queue types that allows me limit traffic per-user basis, so i have to create 2 queue types.
/queue type add name="PCQ_Upload" kind=pcq pcq-rate=0 pcq-limit=50 pcq-classifier=src-address pcq-total-limit=2000 
/queue type add name="PCQ_Download" kind=pcq pcq-rate=0 pcq-limit=50 pcq-classifier=dst-address pcq-total-limit=2000 
now i have to limit all the rates using queue tree, i will limit it that way - that first 2Mbits (as in mangle) will have higher priority over rest of traffic where connection bytes exceed 2Mbits.

i assume that connection that is available have 5Mbits full-duplex
/queue tree add name="Main_Upload" parent=Public packet-mark="" limit-at=0 queue=default priority=8 max-limit=5000000 burst-limit=0 burst-threshold=0 burst-time=0s disabled=no 

/ queue tree add name="Up First 2Mbyte" parent=Main_Upload packet-mark=new_packet limit-at=4000000 queue=PCQ_Upload priority=1 max-limit=5000000 burst-limit=0 burst-threshold=0 burst-time=0s disabled=no 

/ queue tree add name="Up Rest Mbytes" parent=Main_Upload packet-mark=old_packets limit-at=1000000 queue=PCQ_Upload priority=8 max-limit=5000000 burst-limit=0 burst-threshold=0 burst-time=0s disabled=no 

/ queue tree add name="Main_Download" parent=Local packet-mark="" limit-at=0 queue=default priority=8 max-limit=5000000 burst-limit=0 burst-threshold=0 burst-time=0s disabled=no 

/ queue tree add name="Down First 2Mbyte" parent=Main_Download packet-mark=new_packet limit-at=4000000 queue=PCQ_Download priority=1 max-limit=5000000 burst-limit=0 burst-threshold=0 burst-time=0s disabled=no 

/ queue tree add name="Down Rest Mbytes" parent=Main_Download packet-mark=old_packets limit-at=1000000 queue=PCQ_Download priority=8 max-limit=5000000 burst-limit=0 burst-threshold=0 burst-time=0s disabled=no 
any suggestions? comments?
Last edited by janisk on Tue Feb 06, 2007 2:14 pm, edited 3 times in total.
 
User avatar
macgaiver
Forum Guru
Forum Guru
Posts: 1764
Joined: Wed May 18, 2005 5:57 pm
Location: Sol III, Sol system, Sector 001, Alpha Quadrant

Thu Dec 21, 2006 3:25 pm

I was 1day from doing the same thing (make some configuration and paste it in here) - Good work Janis

And what about UDP??? 8) 8) :wink:
 
User avatar
janisk
MikroTik Support
MikroTik Support
Topic Author
Posts: 6263
Joined: Tue Feb 14, 2006 9:46 am
Location: Riga, Latvia

Thu Dec 21, 2006 3:36 pm

maybe someone could do that?
Last edited by janisk on Fri Dec 22, 2006 10:54 am, edited 1 time in total.
 
mmx
just joined
Posts: 12
Joined: Thu Dec 21, 2006 12:07 pm

Thu Dec 21, 2006 3:56 pm

hi,

i'm also very interesting in this advances QoS. It will be gret to find a right solution and to put it like an example in one of manual.

I'll try also to post some scripts
 
User avatar
raivis-v
Frequent Visitor
Frequent Visitor
Posts: 55
Joined: Thu Jul 14, 2005 10:21 am
Location: Latvia, Riga

Thu Dec 21, 2006 4:42 pm

hi,

i'm also very interesting in this advances QoS. It will be gret to find a right solution and to put it like an example in one of manual.

I'll try also to post some scripts
WIKIPEDIA is for examples. If you make some good example script, better place it here: http://wiki.mikrotik.com/wiki/Main_Page
 
jaafarkhalaf
just joined
Posts: 6
Joined: Mon Aug 28, 2006 5:38 pm

Mon Feb 05, 2007 6:31 pm

this is very useful and powerful procedure.....thank u janisk....i applied it to my MT and it works fine :lol:
 
User avatar
samsoft08
Long time Member
Long time Member
Posts: 613
Joined: Sat Nov 26, 2005 10:52 pm

Mon Feb 05, 2007 6:56 pm

does the marking rules mark the PROXY packets ??
i think it doesnt ..
 
User avatar
samsoft08
Long time Member
Long time Member
Posts: 613
Joined: Sat Nov 26, 2005 10:52 pm

Mon Feb 05, 2007 9:58 pm

very important :

from the manual :
connection-bytes=2000000-0 means that the rule matches if more than 2MB has been transfered through the relevant connection
2000000 is 2MByte not 2Mbit...
 
User avatar
janisk
MikroTik Support
MikroTik Support
Topic Author
Posts: 6263
Joined: Tue Feb 14, 2006 9:46 am
Location: Riga, Latvia

Tue Feb 06, 2007 10:28 am

very important :

from the manual :
connection-bytes=2000000-0 means that the rule matches if more than 2MB has been transfered through the relevant connection
2000000 is 2MByte not 2Mbit...
in that field you enter value of - how many bits to count.

byte is 8 bits

kbyte is 1024 bytes and 8192 bits

so do the math how much 2000000 bits are in Bytes or MBytes
 
User avatar
samsoft08
Long time Member
Long time Member
Posts: 613
Joined: Sat Nov 26, 2005 10:52 pm

Tue Feb 06, 2007 1:22 pm

thats right .. but in your post you said 2Mbits which is wrong ..
its 2Mbytes according to the manual..
 
User avatar
janisk
MikroTik Support
MikroTik Support
Topic Author
Posts: 6263
Joined: Tue Feb 14, 2006 9:46 am
Location: Riga, Latvia

Tue Feb 06, 2007 2:15 pm

corrected error in my post, thx for pointing that out.

these really are bytes not bits

then do not mess with M is for 1000000 not 1024*1024 (like in hard-drives)
 
menfung
just joined
Posts: 10
Joined: Tue Jun 05, 2007 10:28 pm

Re: Burst for each TCP connection

Tue Jun 19, 2007 3:02 am

This is little how-to create manual burst using queue tree.

As it is bandwidth control using queue tree first we need to mangle traffic

first i mangle all connections, then i mark first 2Mbytes then i mark the rest of packets
/ip firewall mangle add chain=forward protocol=tcp action=mark-connection new-connection-mark=new_conn passthrough=yes comment="mark all new connections" disabled=no 
/ip firewall mangle add chain=forward protocol=tcp  connection-mark=new_conn connection-bytes=0-2000000 action=mark-packet new-packet-mark=new_packet passthrough=no comment="mark packets" disabled=no 
/ip firewall mangle add chain=forward protocol=tcp connection-mark=new_conn action=mark-packet new-packet-mark=old_packets passthrough=no comment="marking old packets" disabled=no 
any suggestions? comments?

Hi Janisk,


Thanks for this good configuration, it's so cool.

But how to target those mangle markings above to some IP addresses or some Group of IP addresses?? Hopefully you won't mind to teach a newbie like me.


Thks n Rgds,

Men Fung
 
menfung
just joined
Posts: 10
Joined: Tue Jun 05, 2007 10:28 pm

Re: Burst for each TCP connection

Tue Jun 19, 2007 12:20 pm

Hi Janisk,

I did it well... I was confous before, I shared the internet connection to 2 networks, LAN and WLAN, but everything is OK now, I also solve my client's http download problem that caused all the other clients can not browse. Your configuration is really good... once more thks..

And if I have another problem in future, pls give a hand.


Thks n Rgds,

Men Fung
 
trottolino1970
Member Candidate
Member Candidate
Posts: 192
Joined: Thu May 17, 2007 4:25 pm
Contact:

Re: Burst for each TCP connection

Tue Sep 25, 2007 2:39 pm

Hi Janisk,

I did it well... I was confous before, I shared the internet connection to 2 networks, LAN and WLAN, but everything is OK now, I also solve my client's http download problem that caused all the other clients can not browse. Your configuration is really good... once more thks..

And if I have another problem in future, pls give a hand.


Thks n Rgds,

Men Fung
what mean pubblic in configuration of queue tree
 
User avatar
Antixx
Member Candidate
Member Candidate
Posts: 100
Joined: Mon Jun 26, 2006 9:46 am

Re: Burst for each TCP connection

Mon Jan 21, 2008 7:16 pm

what should be in parent if we use PPPOE client instead? because after that we would limit the user in simple queue.
 
carcara
just joined
Posts: 13
Joined: Fri May 25, 2007 3:10 pm

Re: Burst for each TCP connection

Tue Jan 22, 2008 6:42 pm

i have the same problem... probably the default config. All simple is in this.
 
menfung
just joined
Posts: 10
Joined: Tue Jun 05, 2007 10:28 pm

Re: Burst for each TCP connection

Tue Jan 22, 2008 8:12 pm

what mean pubblic in configuration of queue tree
Sorry.. just read this message...

Public is the name I gave to the interface that connected to the ISP.


thks,
 
pospanko
Member Candidate
Member Candidate
Posts: 279
Joined: Sun Dec 18, 2005 4:23 pm

Re: Burst for each TCP connection

Mon Feb 04, 2008 12:30 pm

Why is on Parent Upload and Download queue queue type set to default and not to PCQ_UP/PCQ_DL?
 
pospanko
Member Candidate
Member Candidate
Posts: 279
Joined: Sun Dec 18, 2005 4:23 pm

Re: Burst for each TCP connection

Tue Feb 12, 2008 10:18 am

Anyone...?
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8709
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: Burst for each TCP connection

Thu Feb 14, 2008 2:54 am

Why is on Parent Upload and Download queue queue type set to default and not to PCQ_UP/PCQ_DL?
because PCQ do not affect child queues, so it really makes no matter, what type you set for parent: parent is just to see total traffic
 
pospanko
Member Candidate
Member Candidate
Posts: 279
Joined: Sun Dec 18, 2005 4:23 pm

Re: Burst for each TCP connection

Thu Feb 14, 2008 1:01 pm

Ok. Thx mate!
 
Closed_1
just joined
Posts: 7
Joined: Fri Dec 08, 2006 11:40 am

Re: Burst for each TCP connection

Thu Feb 28, 2008 9:56 am

cool code dude....

but if i had 3 different client group in my office.... each group the ip listed in adrress list.
ie:
ip1 192.168.1.1
ip1 192.168.1.5
ip1 192.168.1.7
ip2 192.168.1.4
ip2 192.168.1.12
ip2 192.168.1.2
ip3 192.168.1.10
ip3 192.168.1.11
group 1 limit traffic by 96 kbps download 96 kbps upload
group 2 limit traffic by 128 kbps download 64 upload
group 3 limit traffic by 200 kbps download 128 upload

each group using PCQ Equal Share with mangle and queue tree writen in wiki
how to implemented those script to my script?

so i must write 2 connection mark? and 2 queue tree for different group, is that right?

another question.
i had idea to traffic prioritizing in each group, which email traffic (pop3)to be higher priority than http traffic. how to implemented like these to PCQ Equal Share in wiki???
 
User avatar
Antixx
Member Candidate
Member Candidate
Posts: 100
Joined: Mon Jun 26, 2006 9:46 am

Re: Burst for each TCP connection

Thu Feb 28, 2008 8:34 pm

Dear Friends,

I have tested in my network and there is a nice problem ;) that may be reason you to dont use this code or you may find the way to solve this problem.

In my situation which i have set the 1024kbps for first 500k and then fall down to 512kbps for special users when a user download a big file after 2 or 3 MB fall down to 512kbps BUT if that person use software like IDM (Internet Download Manager) each time he/she click on STOP and then again START he/she will get 1024kbps speed :D
ofcouse no one call me to tell me because i didnt use this code generally just for test only and I found this :shock:

Please tell me your opinion about this. it wil be nice if we could find the way to block this kind of trick which users may (actually) will use.
 
User avatar
hulk-bd
Member Candidate
Member Candidate
Posts: 231
Joined: Mon Sep 03, 2007 7:19 pm
Location: Uttara, Dhaka, Bangladesh

Re: Burst for each TCP connection

Thu Feb 28, 2008 10:05 pm

Dear friend Antixx,

Yes there is a problem with download managers, cause if clients stops and starts his/her download after every 1mb then it's starts as a freash (1st rule) rate ratio. and with download manager there has few options like 5 connections for a file download that's why the download bandwidth fluctuate very time between 1st and 2nd pcq rule. you can close per IP connection limites with firewall.

Hope you understand that

Thanks
 
User avatar
Antixx
Member Candidate
Member Candidate
Posts: 100
Joined: Mon Jun 26, 2006 9:46 am

Re: Burst for each TCP connection

Fri Feb 29, 2008 5:05 pm

Dear friend Antixx,

Yes there is a problem with download managers, cause if clients stops and starts his/her download after every 1mb then it's starts as a freash (1st rule) rate ratio. and with download manager there has few options like 5 connections for a file download that's why the download bandwidth fluctuate very time between 1st and 2nd pcq rule. you can close per IP connection limites with firewall.

Hope you understand that

Thanks
I think i understand the problem and the reason. but in my view if i limit the connected to each dst ip to 5 connection this problem will occure again? isnt it?
 
titius
Member
Member
Posts: 338
Joined: Mon Oct 17, 2005 11:43 am
Location: Titel Serbia

Re: Burst for each TCP connection

Sun Mar 09, 2008 9:37 pm

if there is wackos to stop start download every minute or so, then let them do that . . .
 
whyaskgh
newbie
Posts: 27
Joined: Tue May 30, 2006 11:30 pm

Re: Burst for each TCP connection

Mon Mar 10, 2008 1:14 am

Please how can this be applied on a mt with load balancing?
Thanks
 
User avatar
janisk
MikroTik Support
MikroTik Support
Topic Author
Posts: 6263
Joined: Tue Feb 14, 2006 9:46 am
Location: Riga, Latvia

Re: Burst for each TCP connection

Mon Mar 10, 2008 12:01 pm

first i would see that user that stops download after each X Mbit you have set in a download.

please read manual about simple queues and queue tree. if you have several groups, then just add tree nodes to the queue tree and set up configuration for them, so, they function properly, you will have to mark connections and then packets for each type of users, you just add dst-address-list for download and src-address-list for upload for each group.

after that you simple follow the example
 
User avatar
Antixx
Member Candidate
Member Candidate
Posts: 100
Joined: Mon Jun 26, 2006 9:46 am

Re: Burst for each TCP connection

Fri Mar 14, 2008 12:51 pm

i think it is misunderstanding, with this setup usercan achive more bandwidth by doing stop and start the download manager software, anyway to solve this problem?

it will be great doing this...

Thank you very much in did/
first i would see that user that stops download after each X Mbit you have set in a download.

please read manual about simple queues and queue tree. if you have several groups, then just add tree nodes to the queue tree and set up configuration for them, so, they function properly, you will have to mark connections and then packets for each type of users, you just add dst-address-list for download and src-address-list for upload for each group.

after that you simple follow the example
 
erjon
just joined
Posts: 16
Joined: Sun May 28, 2006 11:56 am

Re: Burst for each TCP connection

Mon Mar 31, 2008 3:34 pm

Hi to all. I've tested this configuration on my MT 2.9.51 at work. And i must say is wonderful.But the problem is if i start to download with flashget, configured to download with only 1 connections, this setup is perfect.Sow when i configured flashget to download with 5-10 connections, the download speed vary from 1st rule to the 2nd rule after 1 min and after 3 min vary from 2nd rule to the 1st, and so on.So my opinion is to limit at firewall filter to 1/32 connections for the specific packet_mark old.At this point i see another problem, at firwall filter we can drop only syn connections with packet_mark old, what about those with estabilished type, cause if i start to download with flashget with configuration of 10 connections, at this moment in connections tab of MT is see 10 connections estabilished. How can i close 9 of these?
P.s. sorry for my bad english. my configuration is excast as descripted in this thread.
Best regards to all from Albania.
 
erjon
just joined
Posts: 16
Joined: Sun May 28, 2006 11:56 am

Re: Burst for each TCP connection

Fri Apr 11, 2008 1:23 pm

No one can help at this point?
 
titius
Member
Member
Posts: 338
Joined: Mon Oct 17, 2005 11:43 am
Location: Titel Serbia

Re: Burst for each TCP connection

Wed May 14, 2008 2:59 am

Where have all posts gone??!!
 
0ldman
Forum Guru
Forum Guru
Posts: 1465
Joined: Thu Jul 27, 2006 5:01 am

Re: Burst for each TCP connection

Wed Aug 13, 2008 8:34 am

Bump.

I've been using this, works great, however, does TCP have to be identified in the mangle rules?

I'm having a hard time shaping other traffic separately. If I try to use mangle to prioritize port 80 and give other ports a 256k limit, PCQ just stops.

In order to shape P2P, I have to use another RB. I've been doing it on the client side, works well as I only have two problem clients, but I'd like to have a blanket rule so some new customer that discovers P2P won't go completely nuts on the network.

edit: tinkering with my 2nd AP in line, first doing this as the primary, it appears that choosing the TCP protocol is not required. Any input from those more experienced would be greatly appreciated.
 
User avatar
dunga
Member Candidate
Member Candidate
Posts: 254
Joined: Fri Jan 23, 2009 9:51 am
Location: Nigeria

Re: Burst for each TCP connection

Sun Oct 11, 2009 10:22 pm

What of for hotspot users.

I have 3 interfaces,
ether1 = wan- isp
ether2 = hotspot ip range: 172.16.0.0/24
ether3 = local users = ip range : 192.168.20.0/24

How do i go about it
 
lukkes
Member Candidate
Member Candidate
Posts: 177
Joined: Mon Jun 16, 2008 2:12 am
Location: Venezuela
Contact:

Re: Burst for each TCP connection

Fri Jan 15, 2010 7:27 am

Bump.

I've been using this, works great, however, does TCP have to be identified in the mangle rules?

I'm having a hard time shaping other traffic separately. If I try to use mangle to prioritize port 80 and give other ports a 256k limit, PCQ just stops.

In order to shape P2P, I have to use another RB. I've been doing it on the client side, works well as I only have two problem clients, but I'd like to have a blanket rule so some new customer that discovers P2P won't go completely nuts on the network.

edit: tinkering with my 2nd AP in line, first doing this as the primary, it appears that choosing the TCP protocol is not required. Any input from those more experienced would be greatly appreciated.
I have the same problem, i make a lot of rules to mark, ssh,p2p, pop, etc. and make a queue for prioritize it, then mark it in postrouting and prerouting, works perfect, but when i setup this or the Janis mangle example from the mum that marks connections in forward the apply PCQ, the pcq simply doesnt work.. somebody can help us? thanks
 
rmichael
Forum Veteran
Forum Veteran
Posts: 718
Joined: Sun Mar 08, 2009 11:00 pm

Re: Burst for each TCP connection

Fri Jan 15, 2010 7:42 pm

Maybe you are overwriting forward chain marks with postrouting marks?
 
lukkes
Member Candidate
Member Candidate
Posts: 177
Joined: Mon Jun 16, 2008 2:12 am
Location: Venezuela
Contact:

Re: Burst for each TCP connection

Fri Jan 15, 2010 11:11 pm

well, the forward chain marks connection based on src-adress-list but the prerouting chains marks packets based on src and dst ports. alos mark some connection to use the connection-bytes but its marked also by src port 80.. any idea?
 
wifi442
Frequent Visitor
Frequent Visitor
Posts: 81
Joined: Tue Jan 12, 2010 11:01 pm

Re: Burst for each TCP connection

Thu Mar 25, 2010 11:04 pm

Hi to all. I've tested this configuration on my MT 2.9.51 at work. And i must say is wonderful.But the problem is if i start to download with flashget, configured to download with only 1 connections, this setup is perfect.Sow when i configured flashget to download with 5-10 connections, the download speed vary from 1st rule to the 2nd rule after 1 min and after 3 min vary from 2nd rule to the 1st, and so on.So my opinion is to limit at firewall filter to 1/32 connections for the specific packet_mark old.At this point i see another problem, at firwall filter we can drop only syn connections with packet_mark old, what about those with estabilished type, cause if i start to download with flashget with configuration of 10 connections, at this moment in connections tab of MT is see 10 connections estabilished. How can i close 9 of these?
P.s. sorry for my bad english. my configuration is excast as descripted in this thread.
Best regards to all from Albania.

I was having the same problem as mentioned here until I used our new friend connection-rate :-)

I set the connection rate limit for each connection to be just over the connection speed of the non burst queue. It seems to be working so far. If multiple downloads are started with a download manager, each individual connection never reaches the speed specified in connection-rate so they stay in the non-burst queue. Its working beautifully so far.

Here is a small preview :-)
add action=mark-connection chain=forward comment="" connection-bytes=\
    0-2000000 connection-mark=JetWire_Conn_1m connection-rate=2M-100M \
    disabled=no new-connection-mark=JetWire_Conn_1m_Burst passthrough=yes \
    protocol=tcp src-address-list="PPPoE Private 1M Service"

Who is online

Users browsing this forum: Bing [Bot], deatras, haung05, Semrush [Bot] and 91 guests