hi, I have only started to look at VLAN recently . My understanding is that, the switch VLAN only comes into consideration if you wish to have hardware offload for VLAN (the traffic between VLAN ports would be near wire speed because the swhich chip feature support this).
note on the wiki:
For devices that have multiple switch chips (for example, RB2011, RB3011, RB1100), each switch chip is only able to switch VLAN traffic between ports that are on the same switch chip, VLAN filtering will not work on a hardware level between ports that are on different switch chips, this means you should not add all ports to a single bridge if you are intending to use VLAN filtering using the switch chip, VLANs between switch chips will not get filtered. You can connect a single cable between both switch chips to work around this hardware limitation, another option is to use Bridge VLAN Filtering, but it disables hardware offloading (and lowers the total throughput).
so if you cannot fit all VLAN port on the same switch chip, CPU will be involved in forwarding traffic to another switch chip, you might as well use bridge VLAN for it's simplicity .
hope this helps you.