Now that I have a new RB4011 router I am tinkering a bit with the optimization of queueing on my VDSL line.
I have a ZyXEL VMG4005-B50A VDSL modem. I need to do PPPoE-over-VLAN6 with my provider and this modem can do QoS using 802.1p tagging on the PPPoE packets sent from MikroTik to the modem (which operates in bridge mode).
So outgoing packets go like this: pppoe-client -> VLAN subinterface -> ether1 -> VMG4005-B50A -> VDSL line.
When I do a mangle "set priority from DSCP 3 bits" in the postrouting chain, the packets nicely get prioritized when routed to the pppoe-client interface, this priority gets copied into the priority field of the VLAN header, is transferred to the VMG4005-B50A which uses it to select the QoS queue, and I have good QoS on the uplink.
Now I am trying to get this to work with an L2TP/IPsec VPN connection (this router is the client). Unfortunately, L2TP has no "DHCP: inherit" setting so I cannot just copy the DSCP from the payload packet into the L2TP packet (hoping that it would be matched in the postrouting chain and the priority be set correctly).
I thought I may get by when I can skip the set priority the second time the packet goes through postrouting, assuming that it would keep the originally assigned priority of the inner packet. But that does not appear to work. Probably the L2TP layer, unlike the PPPoE layer, creates an entirely new packet (that passes through the output rather than the forwarding chain) and it does not transfer the priority field.
But this is where it gets weird: whenever I try to manipulate the priority of the IPsec (ESP) packets differently from what they were set before being encrypted, the connection cannot pass traffic anymore.
This surprises me... I could understand it when I tried to change the DSCP value of the packets, because they would invalidate the IPsec authentication, but I am not changing anything in the packet when I manipulate the priority, at least I think so...
What could be going on here, and would there be any way to set the priority of this traffic through the entire stack of:
IP traffic -> set priority from DSCP -> route to L2TP/IPsec interface -> route to pppoe-client -> VLAN subinterface -> ether1 -> VMG4005-B50A -> VDSL line.
It does not really matter that the DSCP of the ESP packet isn't copied from the inner DSCP value, as long as the priority (as determined from the inner DSCP value) ends up correctly in the VLAN header.