I am trying to make a configuration that would allow me to divide the WAN1 traffic to three categories (ordered by priority):
- All other internet traffic (inbound and outbound) from and to the Streaming PC (i.e. Online Games, Spotify) [This is determined after filtering out the traffic for item #2 on this list]
- Outgoing traffic from the Streaming PC to Twitch ingest servers (i.e. live-ams.twitch.tv)
- All other internet traffic from and to other hosts on my network
I currently have a script in place that:
- Resolves and populates/cleans-up the address list entries for the aforementioned ingest servers.
- Toggles on/off all mangle rules containing the keyword "LVSTRM".
Then there are some permanent address lists, such us: STREAM which is the streaming PC and LOCAL which contains all of my internal subnets (which comes into play on the first rule, so everything below it are WAN interactions).
/ip firewall mangle add action=accept chain=prerouting comment="Exclude LAN traffic" dst-address-list=LOCAL src-address-list=LOCAL add action=mark-routing chain=prerouting comment="[LVSTRM] Route all internet traffic from the streaming PC through WAN1" new-routing-mark=ToWAN1 passthrough=yes src-address-list=STREAM add action=mark-connection chain=prerouting comment="Mark inbound WAN1 connections" connection-mark=no-mark in-bridge-port=ge0 in-interface=bridge1 new-connection-mark=WAN1 passthrough=yes add action=mark-connection chain=prerouting comment="Mark inbound WAN2 connections" connection-mark=no-mark in-interface=vlan2 new-connection-mark=WAN2 passthrough=yes add action=mark-packet chain=prerouting comment="[LVSTRM] Mark all packets destined to twitch ingest servers through WAN1 from streaming PC" connection-mark=WAN1 dst-address-list=TWITCH new-packet-mark=QOS-GE1-STREAM packet-mark=no-mark passthrough=yes src-address-list=STREAM add action=mark-packet chain=prerouting comment="[LVSTRM] Mark all other packets destined to WAN1 from the streaming PC" connection-mark=WAN1 new-packet-mark=QOS-GE1-STREAM-OTHER packet-mark=no-mark passthrough=yes src-address-list=STREAM add action=mark-packet chain=prerouting comment="[LVSTRM] Mark all other packets destined to the streaming PC from WAN1" connection-mark=WAN1 dst-address-list=STREAM new-packet-mark=QOS-GE1-STREAM-OTHER packet-mark=no-mark passthrough=yes add action=mark-packet chain=prerouting comment="[LVSTRM] Mark all packets destined to WAN1 from other hosts" connection-mark=WAN1 new-packet-mark=QOS-GE0-CATCHALL packet-mark=no-mark passthrough=yes add action=mark-routing chain=prerouting comment="Attach routing marks to already marked inbound WAN1 connections" connection-mark=WAN1 in-interface-list=hosts new-routing-mark=ToWAN1 passthrough=no add action=mark-routing chain=prerouting comment="Attach routing marks to already marked inbound WAN2 connections" connection-mark=WAN2 in-interface-list=hosts new-routing-mark=ToWAN2 passthrough=no add action=mark-routing chain=output comment="Attach routing marks to already marked outbound WAN1 connections" connection-mark=WAN1 new-routing-mark=ToWAN1 passthrough=no add action=mark-routing chain=output comment="Attach routing marks to already marked outbound WAN2 connections" connection-mark=WAN2 new-routing-mark=ToWAN2 passthrough=no /queue tree add max-limit=8800k name=GE0-TX parent=ge0 queue=default add max-limit=55M name=GE1-RX parent=ge1 queue=default add limit-at=6250k max-limit=6560k name=GE1-STREAM-TX packet-mark=QOS-GE1-STREAM parent=GE0-TX priority=2 queue=default add limit-at=1290k max-limit=1430k name=GE1-STREAM-OTHER-TX packet-mark=QOS-GE1-STREAM-OTHER parent=GE0-TX priority=1 queue=default add name=GE1-STREAM-RX packet-mark=QOS-GE1-STREAM parent=GE1-RX priority=2 queue=default add name=GE1-STREAM-OTHER-RX packet-mark=QOS-GE1-STREAM-OTHER parent=GE1-RX priority=1 queue=default add max-limit=55M name=GE0-RX parent=global queue=default add name=GE0-CATCHALL-RX packet-mark=QOS-GE0-CATCHALL parent=GE0-RX priority=3 queue=default add limit-at=400k max-limit=480k name=GE0-CATCHALL-TX packet-mark=QOS-GE0-CATCHALL parent=GE0-TX priority=3 queue=default
Now onward to the actual problem(s):
- WAN1 is a 100/10Mbps VDSL connection and while my hAP AC can handle the upstream prioritization/shaping and everything works perfectly.. the problems start when someone utilizes the downstream at full blast. The CPU utilization immediately spikes and remains at 100% and significant amounts of packets start to drop from the queues.
- If the high CPU utilization is sustained for long enough, watchdog will kick in and reboot the device. (Has already happened 5 times in 3 days)
This is what MikroTik support had to say about the watchdog reboots:
Each parent queue (with child queues if there are any) works on single CPU core. All packets that must be processed by this queue go through it in the order and if such queue manages to load single CPU core to 100%, then other cores must also wait for these packets to be processed since packet processing is the highest priority task on router. At that point other CPU cores gets stuck on 100% (basically on waiting state) and as a result single queue may overload router completely. Such situation may lead to a Watchdog timer.
And some questions:
- Is there a way I can do this a little bit more efficiently and overwhelm the device less, besides limiting the overall downstream to 40Mbps? I clearly must be doing something wrong, there's no way the device can't handle more than that.
- Why on earth isn't watchdog prioritized over packet processing? I believe it's more catastrophic if the device reboots, than to lose/delay a packet or two.
PS: Stroke through some information that might've been invalid or irrelevant.