What is the recommended way of shaping traffic with queue trees when the outbound interface is not static? If I'm understanding the flows correctly, there's no way to use queue trees to shape upstream and downstream communication to different rates without doing so at the outbound interface for each stream from the router. While limiting a host's downstream is simple, as I intend to perform the shaping where the router connects to a switched network with a static interface and route, upstream is more difficult, as the traffic may route out of the router over multiple interfaces.

And with OSFP maintaining my routing and giving me redundancy should a link fail, I don't necessarily have an interface that I can specify as a upstream interface -- it may change on me, traffic may route across multiple interfaces depending on the destination, etc.

As I understand it:

1) Queue trees can only limit traffic at an interface, which prevents me from doing the shaping, say, at some point in the middle of the routing decision like the forward chain. This means I can only limit traffic where it actually enters or exits the router's packet flow decision table.

2) In addition, queue trees need to have a packet mark applied to traffic to do any sort of decision about what traffic to shape and how to shape it. This means that it /has/ to be done at the outbound interface, as there's obviously no way to mark traffic before it actually comes in on the interface.

3) I can't use global-out as the outbound interface, because this would apply to the interface connected to the switched network as well, causing my upstream queue to limit downstream connections as well.

Given these three facts, the only way I can see is to copy my queue tree for limiting upstream onto every possible interface that traffic might be taking as an upstream route. This isn't very elegant, and will probably be a PITA to maintain. Is there some way to define a queue tree as applying to all interfaces save specified ones?

Or am I missing something, such as some way to perform bi-directional shaping for traffic passing through a single interface, allowing me to ignore the fact that I don't know what route each individual traffic stream is going to take out of the router?

well, (un)fortunately, you're almost completely wrong. first, see http://wiki.mikrotik.com/wiki/Packet_Flow

1. it depends on Parent. Global-In do shaping even berofe Routing Decision

2. yes, packets are grouped by marks. no, you may mark packets everywhere. if you do it in prerouting - they are marked just after the inbound interface, again before Routing Decision

3. Global-Out limits packets, not connections. if you mark only upload packets - it will limit your upload; if you mark every 2nd packet - then you will limit even packets, odd packets being passed not limited; etc. =)

something like this

Might be a bit of language barrier here, cause I don't understand anything more about the shaping logic after that post. I'd looked over the wiki exhaustively before posting here; I'm well aware of the packet flow diagram. Where in the packet flow does the shaping actually occur, then? Routing decision?

1. it depends on Parent. Global-In do shaping even berofe Routing Decision

1) Queue trees can only be assigned to interface parents; and global-in/out. I took that to mean that that's where the shaping logic comes into play. Yes/no?

2. yes, packets are grouped by marks. no, you may mark packets everywhere. if you do it in prerouting - they are marked just after the inbound interface, again before Routing Decision

2) Prerouting still happens AFTER the input interface, so therefore a shaping system attached to a specific interface won't be able to operate on packet marks, as they'd get marked after the packets pass through the interface. Yes/no?

And what does 'I may mark packets everywhere' even mean? I have to do it at a specific chain -- forward, input, output, prerouting, and postrouting. Till I hit one of those chains I can't get a mark onto a packet.

3. Global-Out limits packets, not connections. if you mark only upload packets - it will limit your upload; if you mark every 2nd packet - then you will limit even packets, odd packets being passed not limited; etc. =)

3) s/connections/packets/. Connections are made up of packet streams, which is why I used the term. I'm still limiting the individual packets, yes. The gist of my statement was not that I was limiting full connections, but that global-out would match all interfaces on the system. Here, I'll restate it:

I can't use global-out as the outbound interface, because this would apply to the interface connected to the switched network as well, causing my upstream queue to limit downstream packets as well.

something like this

Ugh, I really don't want duplicate rules.

I'm marking packets via connection working from an address list, then marking the resulting packets, as per so-called best practices on the wiki. Is this what's causing my issue? This only gives me one mark for both directions of a connection. I'd expect when using iptables to simply need to mark the packets w/ a dst address and src address, get individual upstream/downstream marks, and shape the traffic using those.

1- Only leaf classes are able to store packets, so, only leaf classes are able to shape traffic with a correct queue discipline.
2- Incoming traffic can be shaped by marking packets on prerouting chain with an attached queue on global-in for that mark.
3- Only if you mark it. You may set global-out for upstream as well for downstream, commonly used to group pppoe interfaces. It does NOT implies that upcoming traffic will be queued together with outgoing traffic, but only if both are being marked of course.

So far, what you want is completely possible and you may just need to figure it out. Otherwise, why don't you simply post your rules? Then we could eventually post back any necessary fixes.

1- Only leaf classes are able to store packets, so, only leaf classes are able to shape traffic with a correct queue discipline.
2- Incoming traffic can be shaped by marking packets on prerouting chain with an attached queue on global-in for that mark.
3- Only if you mark it. You may set global-out for upstream as well for downstream, commonly used to group pppoe interfaces. It does NOT implies that upcoming traffic will be queued together with outgoing traffic, but only if both are being marked of course.

2. Right, but if you mark connections then use that as a basis to mark packets, you can't differentiate upload traffic from download traffic on global-in. I'm pretty sure that following this 'best practice' is what's causing my issue.

So far, what you want is completely possible and you may just need to figure it out. Otherwise, why don't you simply post your rules? Then we could eventually post back any necessary fixes.

I'm using the ruleset outlined in the QOS best practice presentation on the wiki (http://mum.mikrotik.com/presentations/C ... _Megis.pdf). The problem is that it assumes you can assign traffic to set inbound and outbound interfaces on the router, instead of assigning everything based off a single interface connected to a POP's switched network. That combined with both directions of the stream having the same mark due to marking connections and then packets mean that you have to sort traffic into upload/download categories based solely on the interface the traffic is routed on.

As I mentioned in my previous post, I can get around this by making two packet mark mangles (one upstream, one downstream) instead of a connection mark mangle and a packet mark working off the connection mark. Because I then have seperate packetmarks for each direction of traffic, I can work entirely off of those packet marks and not care about what interface the traffic is travelling on.

I suppose I'm asking, why is basing your traffic shaping off of the interface the traffic is travelling on instead of off of some other property of the traffic considered best practice?

Edit: Rules I'm using are in documentation as well; http://www.mikrotik.com/testdocs/ros/3.0/qos/queue.php. I've obviously got different names, and I'm using custom pcq queues, but the basics are the same.
Mark connection, use that to mark packet, create parent 'upload'/'download' queues off specific interfaces, create tree queues as children of those global queues.

1) Queue trees can only be assigned to interface parents; and global-in/out. I took that to mean that that's where the shaping logic comes into play. Yes/no?

yesss. and you may shape traffic twice inside one router, see http://mum.mikrotik.com/presentations/C ... _Megis.pdf

1) Queue trees can only be assigned to interface parents; and global-in/out. I took that to mean that that's where the shaping logic comes into play. Yes/no?

yesss. and you may shape traffic twice inside one router, see http://mum.mikrotik.com/presentations/C ... _Megis.pdf

Um, I have. I linked it in my previous post. I also posted an example out of the 3.0 documentation of a tree queue configured like that PDF outlines, and what my issue with it is.

Edit: Actually, lets make this simple. Given the ruleset outlined in that pdf, what should I set the queue tree entries 'Total_download' and 'Total_upload' parents to on a router where I have ether1 connected to a switched network containing customer access devices and ether2, ether3, and ether4 connected to my network, where traffic from customers may be routed from ether1 to either ether2, 3, or 4 depending on destination of the traffic and current network status as OSPF compensates for any potential downed links. Thus, downstream traffic to a customer routes from any interface to ether1, and upstream routes from ether1 to ether2, 3, and 4.