After looking into several documents I found that Small Business accounts ABSOLUTELY could remove the Google Router and go straight to their own router.
The install I was working on was residential so Google refused to “click that switch” or help me get around the need for their router. https://support.google.com/fiber/answer/6078065
After some more digging I found a 2 years old HOW TO that stated… "Add a VLAN Tag of 2 to your DHCP client.
Well, I wouldn’t be that sure on what gents there may have created - as they don’t use just the VLAN IDs to discriminate between services and need the CoS marks as well, I could theoretically imagine that two DHCPDISCOVER messages coming from the same MAC address and with the same Vendor ID etc. may be served by different DHCPOFFER depending on the different CoS marks they arrive with. So we cannot say whether it is directly the CoS value of 3 that chooses the traffic shaper profile on upload or whether it is the IP address assigned from a different pool.
If the CoS mark was only important in the DHCPDISCOVERY (and DHCPREQUEST) packets, it would have a positive impact on the CPU load because if you need to set CoS in every output frame (which is what I assume to be the case, but I’m on the same continent like @msatter so I cannot check that), you cannot use fasttracking, and even if you don’t use fasttracking for some other reasons, setting CoS is still one action more to be taken on an egress packet so it costs some CPU cycles. I think some CRS handle that in hardware but I think we are discussing a 3011 here, right?
This advice would only make sense to me if the priority should be set only in the VLAN CoS field of the outgoing DHCP packets (DHCPDISCOVER, DHCPREQUEST) but not of the packets routed from other hosts. However, the DHCP client is actually bound closer to the wire than the ip firewall, so /ip firewall mangle chain=output rules never match on outgoing DHCP packets.
So this makes you Lucky Luke, because the switch chips of 3011 are QCA8337 which bless us with hardware manipulation of frame headers.
To set the priority field of the VLAN tag of all frames sent by the CPU and tagged with VLAN ID 2 towards the MAC address of google’s gateway in VLAN 2 to the magic value of 3, regardless whether the IP packets they carry are generated locally (like the DHCP ones) or routed from LAN, and also if they carry ARP packets, you need to do the following:
The above is true if the uplink is really connected to ether6-google as you’ve stated in your OP; if you connected the fibre to an SFP module inserted directly into the 3011, you wouldn’t be able to benefit from the switch chip features as the SFP cage is connected directly to the CPU, bypassing the switch chip. So in that case, you’d have to make the SFP interface a member port of a bridge and use /interface bridge filter rules instead, which would make the WAN processing more CPU costly. If the frames carrying DHCP packets (or ARP packets) need not have the vlan priority field set to 3, /ip firewall mangle add chain=postrouting action=set-priority new-priority=3 out-interface=GoogleVLAN is sufficient as it covers both the locally originated IP traffic (except DHCP) and the IP traffic routed (and NATed) from LAN. But ARP is not IP traffic so it this rule won’t set the priority field in frames carrying ARP packets either because it won’t ever see the ARP packets.
EDIT: I’m not sure now why but the miracle above actually doesn’t happen because the priority value is not being set for frames which ingress the chip via its cpu-facing port; I’ve seen the rule to work for the ethernet ingress ports, but when sniffing what the switch chip has forwarded from the cpu port to an ethernet port on a device connected to that ethernet port, the priority has to be set using the /ip firewall bridge filter or /ip firewall mangle to be seen at the sniffing device. The switch chip rule doesn’t overwrite it.
EDIT 2: matching on vlan-id on the CPU port seems to be the spoiler of the miracle. I’ve tried to replace the vlan-id matching in the original rule by different match condition serving the same purpose in my environment and the rule started matching and setting the priority field. I guess it has something to do with the proprietary format of the frame header which is used between the CPU and the switch. In my case, I was matching a single dst-mac-address because I have more VLANs on the bridge and port; in your case, as the /interface ethernet switch rule don’t allow to match on the egress port, it might make sense to match only the first three bytes of the gateway’s MAC address, i.e. the vendor ID. Also, I haven’t tested whether /interface ethernet switch rule works the same even if the egress port is not a member of any bridge in ROS (i.e. when the /interface VLAN is attached directly to ether6 as in your setup).
EDIT 3: it cannot work either as the DHCPDISCOVERY goes to a broadcast MAC address. And as the source MAC address is the same for all VLANs, matching on src-mac-address may be problematic too.
Follow up from Support…
Sorry, it is my mistake, the forum rule will be the correct one for traffic trough the router. Mine is for traffic sent from the router itself.
So did you have a chance to check whether it is necessary to set the VLAN priority field to 3 also in the DHCP packets to gain the full upload bandwith or whether it is sufficient to do that on the “real” traffic?
Not sure if you’re still trying to figure this out or not, but I did get this to work for me (and to anyone who comes across this problem).
You don’t want to use the output chain because it’s not covering traffic going out of WAN, just ones originating from the router itself via the mangle ruleset.
You want to use the postrouting chain so it hits all packets going out WAN, not just ones generated locally.
This allows me to not need an ethernet switch rule anywhere (I’m uplinking via SFP+ on an RB4011, so that wouldn’t work anyway), and allows me to achieve full gigabit symmetrical speeds.
