how can i combine between two lines in the way of pcc
and in the same time distinguish between browsing and download?

thanks in advance,

What do you mean by combine? PCC is a load balancing set up, not a bonding solution, this means that a given connection will only be able to consume the bandwidth available on the line it is going out of. Bonding, in most cases, is not possible to do on the WAN side of things unless you have multiple connections from the same ISP and they are willing to set up bonding for you, or you have another device at some other location with more bandwidth to bond to.

As for the distinction between web browsing and downloading, as far as the router is concerned and knows, they are one and the same thing. An HTTP page download and an HTTP download for an ISO file are exactly the same thing, so there is no real way to make a distinction between them. You can classify stuff based off of the connection bytes that have been transferred to make a distinction, but that is as close as you are going to get to "seeing" a difference between a client web browsing and downloading a file via HTTP. Also as a note, the router has no way of knowing how much data will be transferred with any given connection, it only knows that after a connection has been closed.

dear feklar .
thanks alot

i reached your point of view but ......
concerning (distinguishing browsing and download) we can ...
i use this way " by the member (hazem amer)" and it works very very well
this way depends on layer7protocols in identifying the content of the page
it depends on the extentions of files we download
i put the rule in an attachment please check it
it is for version 3.3 or more that have layer7protocols

concerning pcc i meant load balancing thanks and sorry for that fault
i know how to carry out loadbalancing pcc

my issue here >>>>>>>
what is the (rule/way) through it we can carry out loadbalancing and in the same time we carry out distinguishing between browsing and download??

in the file attached before

the queue used is queue tree
the limit of download is 1 mega
if the line band width 2 mega then 1 mega for download
2 mega for browsing if download traffic isnt consumed
1 mega for browsing if download traffic is consumed
so that there will be always free space for browsing


by the member hazem amer
imported to benefit

I don't use lay7 personally because it eats a lot of CPU time for it to inspect each packet for it. You are also going to have a very hard combining these two and keeping CPU usage down, PCC uses connection marks and the marchers you have there use connection marks, you cannot mark a packet with two different connection marks, it will mess things up. So you need to modify the rules in such a way that only one uses connection marks, or combine them somehow.

You can try adapting PCC to mark for routing directly and bypass marking the connections, but I don't know how well that will work.

You can try bypassing marking connections with the rule set you listed and mark the packets directly instead of marking the connection and marking packets based off of the connection mark, but you will eat up a ton of CPU time doing it that way.

You can try combining the PCC marks with the connection marks in the listed file, but that is a lot of mangle rules, and that will eat up a lot of CPU time.

thanks for paying attention
but no matter of cpu as my server specifications are high

i want the loadbalancing work with my rule of distinguishing?

Any of the listed methods should work, but you are going to have to play around with modifying the rules a fair amount to get it going yourself. Probably the easiest to try and set up would be the PCC rules without connection marks. Basically replace the mark connections rules with mark routing, and remove the current routing mark rules.

I can't say how much CPU time will be taken up, only after setting up the rules and testing it will you be able to determine that.

no matter for the cpu
my pc specifications are high

any idea about load balancing and distinguishing between download and upload to work in the same time>

any attempts for the answer

What do you mean distinguish from upload and download to make them work at the same time? In most cases, replies to traffic going out of one interface must come back in on the same interface, otherwise the router will see the reply traffic as invalid and it will drop it. A request for a web page is upload traffic as much as someone uploading pictures to the internet. Data is data to a router.

i meant that
i want (the rules of load balancing and the rules i attached before concerning isolate download and browsing) work together


because in my case if i make load balancing , the rules of download isolation arent applied

and thanks for paying attention

By using connection marks and marking packets based off of the connection mark, you automatically get both sides of the connection. If you mark for packets directly and bypass connection marks you need to double up on the rules to get both sides (in the forward chain mark packets with a dst-port of 80 and a src-port of 80).

Then you just need to set up your queues to take all of that into account. I like to place my queues on the interfaces themselves. Your WAN interface(s) is going to be the Upload Queue, and your LAN interface(s) are going to be the Download Queue. This is because the router cannot control what it gets on any interface, it can only control what it sends out of the interface.

Hi Feklar,

What about having a connection marker in the the prerouting chain? Then according the conn. mark next rules give route marks. Still in prerouting chain. Routing decision is now made by routing based on the routing mark of the connection.

Now next rule will be in ´forward´ chain. Here again on different classifiers connections are marked, followed by a package mark which is then used in the Queue tree for Qos?

My argument:
Since the connection is already routed follow its first marking in the prerouting chain, after that same connection just get new connection marks in the forward chain and can then be used for other purpose?

In both instances I filter on the incoming (=LAN or local) port of the router since here the first connections is made by client.

step 1. client initiates connection in conn. tracker
step 2. mangle prerouting filter gives it a conn. mark
step 3. mangle prerouting give this connection marked traffic a route mark
step 4. router takes route decision based upon this route mark
(so traffic now already knows its WAN port to leave the router, return will then come back fm same WAN port and doesn't need to be route marked again since only one LAN interface for all traffic. new package coming from LAN are just marked again by the same step 1 and get same treatment again. I also use PCC here on a rb1000 and to me it looks fine so far.)
step 5. package now hits a new mangle filter in the ´forward´ chain and its connection mark gets overwritten by this filter.
step 6. next filter looks for this new conn. mark and gives package mark to all packages of this connection.
step 7. Queue tree looks for these new package marks and drop them if necessary. For upload Queue tree has a total limit slightly lower than WAN maximum possible upload (85-90%)
step 8. Return package from that connection enters router through same WAN connection was made, and is regognized by conn. tracker as belonging to this connection.
step 9. Queue tree with parent the local interface now looks at the package mark of this connection which is still the one of step 6. and drops it if necessary. Limit-at and Total-Limit are not set of combined download totals for all connected WAN connections.

step 10. is actually step 1 again if traffic for this connection is sending ack's or related traffic to same remote host again.

Since I use PCC with srce-address only connections made from one local IP will always get the same route in this process and therefore we (should) have no broken links.

Do you agree with me so far or do I overlook something here?
I am still working on this setup, it looks like to work although I still see one of the WAN is hardly getting any traffic.

@WirelessRudy: I tried that when i was playing around with QoS on some test boxes here in the office. It doesn't work. The two systems mess each other up by doing that. I thought basically the same thing you have listed, but it's not how it worked out.

hmm, well. what I can try is to do the PCC only with direct routing markers. See if traffic loads better over the 3 WAN's I have.
my rb1000 should be powerfull enought to handle that.

Assigning routing mark directly with PCC works fine. The only problem is that it uses a lot more CPU (3x more) as compared to giving connection mark to new connections using PCC and assigning routing mark based on connection mark.

You'll not get any different connection distribution however (which is a good thing since once of the PCCs features is consistency).

Search for threads discussing using different PCC classifiers per connection type to help load balance more evenly.

PCC runs now with only package marking fine. CPU still below 10% on average and only 10-25% during some peaks so that is no problem.

I use src address only for PCC since any other option creates broken links on authentication session for several sites (AOL-mail, google, some bank sited).
Meaning that with only 30-40 users running through this 3 line PCC setup will not really give good load balancing on average.
The only way PCC creates stable links is a setup that is almost the same as manually distributing each client with an individual routing mark to one of the 3 WAN ports.

I use IP src and dst classifier and it's still not evenly distributed. Maybe in your case it'd be better to use nth instead of pcc and to shorten connection tracking timeouts?

EDIT: just realized nth will not work in your situation...

Basically we have two sorts of load balancing.
1. Load balancing on ALL traffic. Even coming from one IP.
2. Load balancing for the network traffic.

In 1. LB would mean that even one user's traffic request would evenly distributed over all available links
In 2. LB would mean that the usage of the network should as best as possible be evenly distributed over all available links but each single IP only uses one WAN (at the time).

In the first case you could get high though-puts for single user (use of several WAN's for one download. Program is needed that can open multiple connections to file server.) and a very good overall LB. But it will be very hard, not to say impossible, to stay out of the ´broken-link´ issue.

For client overall satisfaction (clients like high throughput's, but it definitely puts them off if their authentication session are broken off very regularly) it is best to go for option 2.
While for network operator that wants to make best use of available capacity option 1 is more desirable.

As long as each WAN connection has enough capacity to fulfill some clients at the same time with their speed demand option 2. is the best. Client get stable network and chance of overload of one line while other is idle is not too big.
Obviously, the more capacity you have on each WAN, also compared to the client's assigned speeds, the better. And the more clients for the LB system the better the overall load balancing.

The only other way of combining best of both options would be to identify which websites need clients to log in always from same IP, even if multiple and or simultaneous http and https sessions are made.
Then you can filter these out and give them src-address only PCC while for all other traffic src and dst address+port combination can be used.
But this sounds like a daunting task to me.

The only definite solution is to have just one big pipe that is big enough to serve each client at the maximum speed and not to have over subscribed your capacity too much. (1/10-15 is a good ratio.)

Actually the discussion on load balancing (PCC, ECMP, Nth) is of same nature as on file download versus browsing.
A router can never now before hand if a connection is made for just browsing to a page or downloading a big file by http.

Same for ´broken link´ issues.

A router can never know before hand if a local host starts a connection with the aim to use one GW only for a consequent client-IP to remote host connection or that local host has the intention to open as many different connections over different GW's to remote host as possible to increase total band with.