A = Internet gateway, 8Mb download.
B = some router
C = AP
D = Client of AP.
Traffic shaping of client happens in C.
First, client has no limit set in queue, client receives requested download with almost 10Mb.
I monitor his download stream in A.
Now, I set client queue to limit download to 3MB. Clients download falls back to 3Mb. >>> normal.
But, in A I also see the download fall back to 3Mb?
Traffic shaping/limiting happens by dropping packages I understood?
So, if feed (adsl line) is capable of supplying 10Mb data stream (proven by unlimited download) but in A no shaping takes place,
why is the flow in A still dropped to a level set in C?
You would imagine only in C packages are dropped and/or re-queued for sending out to D? A and C don’t communicate their settings?
Is it not like water hose: high pipe delivers 1000liters per minute but main valve (A) cannot be shut, so to prevent end user (D) flooding we just open pipe before D (at C) to let some water out. Now client gets less water but flow in A is still the same…
But I presume I understood things wrong here because I see something else happen.
It looks more like I squeeze a valve in the pipe near D (so at C) and since water has no way to go also flow at A is now regulated.
Anybody that can explain a bit more about this process?
Well, in this specific occasion I was playing with a simple queue setting.
The traffic steam was generated on a usenet user, so tcp port 119.
But your answer means in fact it doesn’t really matter where you actually are shaping (queueing) a tcp flow.
But what about udp? Lots of P2P programs use udp, even to transport big volumes of data.
(I have several considerable data flows with udp protocol running fm and to user.) According this article: http://www.tech-pro.net/tcpip2.html they are send anyway, so to stop using too much network resources udp upload should be shaped at client, and download at border gateway? Am I right?
[New questions: What, and how, does simple queue shape traffic. Just ALL packages, independent of protocol? In Queue tree we mangle first so we know what we use later in Queue. But in simple queue we filter nothing first…]
PS.
I increased your karma, your posts are really appreciated!
yes, if you generate UDP stream - you will see whole stream on A and B. then C will shape it. that’s why UDP in p2p networks is evil =) in that case programs should control the bandwidth by themself
simple queues catch packets only by addresses - yes, all packets (until you use packet-marks in simple queues )
Depends on what you stick into the simple queue. If you do it just by target address, all packets (regardless of protocol). If you go by packet mark, all packets marked (and the marks could be applied based on protocol).
It’s generally best to shape the traffic as close to the source as you can if you know how you want to shape it (e.g. policing, rather than prioritizing when there’s congestion), that’s the border router for downloads and the CPE for uploads. What’s the point of transmitting traffic into your network that you will drop later on in your network? But there are tradeoffs to consider - if it’s really cheap for you to backhaul lots of traffic from the CPE to a central point, and if you can save a lot of money by buying dumb CPEs that can’t police, then it makes more sense to apply policies at the central point.
To bad, I’m not giving away karma points that easy, Chupaka was just a bit faster!
His answer was also more to the point because I already wrote I was not filtering or marking anything in this specific case…
I don’t exactly what you mean with “how you want to shape it (e.g. policing, rather than prioritizing when there’s congestion)”
Can you explain? Policing is something to do with routing, shaping is bandwidth throttling and prioritization is increasing quality and securing in-time delivery for certain traffic above other, lesser important, traffic.
But shaping by policy I am not clear what you mean with that?
If you set up mangle to filter so you then can shape traffic, than at the same time you limit you do prioritization as well. Its all happening in the queue tree. Even for the client it is better to have QoS done for his traffic stream so he can still make his voip call or browse while his P2P is running.
Anyway, I thought to have it all under control but now am re-designing my QoS strategy. It works but can always be better.
Sorry, I was using general ideas and Cisco terms. Should have been more explicit. So below an explanation, as I think that even though the implementation is wildly different the ideas are very useful:
In their parlance, ‘policing’ refers to enforcing a rate limit even when you could transmit more traffic. An example would a customer whose CPE is on a Fast Ethernet link that could transmit 100Mbps full duplex, but you sold him a 10/2 line - there’s no congestion, but you’re going to rate limit traffic at those values (with a potential burst).
‘Prioritizing’ is what you’re already used to - it describes the likelihood that a packet will be discarded due to congestion, when you have to discard packets because you cannot transmit then (either due to an artificial rate limit or because actual link capacity is exceeded) priority packets are more likely to still be transmitted.
‘Shaping’ refers to avoiding artificial congestion by an upstream. Let’s say you yourself have a 50/50 circuit from a WAN provider, but it’s delivered on Fast Ethernet. The interface itself won’t ever be congested since it can transmit 100/100, but then the upstream would discard traffic for you and you don’t get to be involved in the decision, so you shape the traffic down to 50/50 yourself.
Policing makes sense to do as close to the CPE as possible, ideally on the CPE itself. You know exactly in advance what you want to do to the traffic. You can be nice and also run priority on the CPE to deal with how likely certain protocols are to be dropped due to the rate limit, but - and this is just my opinion - it’s the customer’s problem and not yours, and priority on the CPE should be sold as an extra service. If they call that their VoIP traffic is dropping and you show them a graph that little Johnny is running P2P and saturating their connection, competing with Dad’s VoIP call you’ve delivered what you promised. Also, each customer will probably have different expectations of what traffic should be prioritized. Again, just an opinion.
Where priority really is important is on the border router and potentially the core, where all customer traffic combined is exceeding link capacity or where traffic directly outbound to the WAN needs to be shaped because you’re maxing out the WAN bandwidth that you have available. You can’t do anything to prepare for this on the CPE other than set DSCP/TOS if that’s part of your service model, as you cannot know at that point what link conditions are further upstream.
If you do prioritize on the CPE, it does make sense to use DSCP/TOS to indicate for the upstream what traffic should be prioritized there. Why spend a lot of time making QoS decisions only to have them negated later by a router two hops up? Also, those later routers are usually busier, and evaluating DSCP/TOS is often cheaper than evaluating in detail the source, destination, protocol and port.
Well, you see. Karma points earned now! Very useful text you wrote. I bet more readers will get some more ideas from it… lets go a bit further into your story:
IMHO this is what every serious ISP, small of big, is dealing with. Customers will never have the full capacity of what you as ISP pay for, unless the customer pays the full bill! So customers get less, and then you can also sell more bandwidth then you actually have. Otherwise you have no business model. But now combined customers at peak times can generate more traffic then your supply can deliver. Now QoS comes in place.
‘Prioritizing’ is what you’re already used to - it describes the likelihood that a packet will be discarded due to congestion, when you have to discard packets because you cannot transmit then (either due to an artificial rate limit or because actual link capacity is exceeded) priority packets are more likely to still be transmitted.
Yep.
‘Shaping’ refers to avoiding artificial congestion by an upstream. Let’s say you yourself have a 50/50 circuit from a WAN provider, but it’s delivered on Fast Ethernet. The interface itself won’t ever be congested since it can transmit 100/100, but then the upstream would discard traffic for you and you don’t get to be involved in the decision, so you shape the traffic down to 50/50 yourself.
What would happen if this would be the situation and your clients demand/deliver more then the 50/50 and I am not shaping? How is traffic going to be dropped? At random? Fifo? Can it be mainstream ISP or carries doe their own QoS on the end points too? Its an presumption because I DO shape but what if not?
Policing makes sense to do as close to the CPE as possible, ideally on the CPE itself. You know exactly in advance what you want to do to the traffic. You can be nice and also run priority on the CPE to deal with how likely certain protocols are to be dropped due to the rate limit, but - and this is just my opinion - it’s the customer’s problem and not yours, and priority on the CPE should be sold as an extra service.
I agree. I decided to go for central client shaping some months ago but actually find more and more arguments to at least limit up and download at client. Alternatively I can do it in the AP. It would also prevent any broadcast storms by abusive software in the AP network if any… and stops the udp upload from P2P users.
If they call that their VoIP traffic is dropping and you show them a graph that little Johnny is running P2P and saturating their connection, competing with Dad’s VoIP call you’ve delivered what you promised. Also, each customer will probably have different expectations of what traffic should be prioritized. Again, just an opinion.
But one I fully share.
Where priority really is important is on the border router and potentially the core, where all customer traffic combined is exceeding link capacity or where traffic directly outbound to the WAN needs to be shaped because you’re maxing out the WAN bandwidth that you have available. You can’t do anything to prepare for this on the CPE other than set DSCP/TOS if that’s part of your service model, as you cannot know at that point what link conditions are further upstream.
If you do prioritize on the CPE, it does make sense to use DSCP/TOS to indicate for the upstream what traffic should be prioritized there. Why spend a lot of time making QoS decisions only to have them negated later by a router two hops up? Also, those later routers are usually busier, and evaluating DSCP/TOS is often cheaper than evaluating in detail the source, destination, protocol and port.
The last words of this paragraph are most valuable. I didn’t look at it this way. My problem is only, how to separate ´urgend´ traffic from ´normal´ and then also find ´bad´ traffic. Last one is the crux. I am talking P2P here. Even with MT’s P2P classifier and the combination of the layer7 filter (one after the other) I still find lots of ´unknown´ traffic. So I can have a filter catching just “everything else” but then have the need to filter out first all ´known´ traffic. Before you know you have a system with many filters. All traffic has to travel through these filters and that is not doing any good at all to the overall performance…
What would happen if this would be the situation and your clients demand/deliver more then the 50/50 and I am not shaping? How is traffic going to be dropped? At random? Fifo? Can it be mainstream ISP or carries doe their own QoS on the end points too? Its an presumption because I DO shape but what if not?
There’s no way to tell (or at least to generalize) because you are not the one dropping traffic, and every ISP might be treating you differently. From your router’s point of view it’s putting less traffic on the interface than the interface itself can support, so everything should be going through, but your ISP is ‘policing’ = rate limiting you, on their CPE or the link to their CPE, just like you are rate limiting your own customers.
If you want to know for sure, ask your ISP.
The last words of this paragraph are most valuable. I didn’t look at it this way. My problem is only, how to separate ´urgend´ traffic from ´normal´ and then also find ´bad´ traffic. Last one is the crux. I am talking P2P here. Even with MT’s P2P classifier and the combination of the layer7 filter (one after the other) I still find lots of ´unknown´ traffic. So I can have a filter catching just “everything else” but then have the need to filter out first all ´known´ traffic. Before you know you have a system with many filters. All traffic has to travel through these filters and that is not doing any good at all to the overall performance…
To reduce load on core and border routers it can definitely make sense then to make those decisions on the CPEs or APs, to mark DSCP/TOS and to simply obey those marks on the core and border, which makes for a very simple rule set on them. The individual CPE/AP CPUs aren’t very fast compared to the ones in the big routers, but they all deal with significantly less traffic at once. And their combined processing power in any medium or larger network is probably going to be more than your core and the border offer.
If you do start using DSCP/TOS just be aware that some smarter customers may use this to cheat, using their own equipment to set high priority marks on all their traffic before putting it on their network. It may be necessary to have a very early rule that resets everything coming from them to normal, and to then classify yourself on the normalized traffic.
)