If you follow VLAN switching manual (the Other devices with built-in switch chip part), you should be able to get it done. And this would be the most device-friendly way.
It might be easier to implement functionality using bridge vlan filtering stuff (as described in this fine tutorial) and in your particular case you might even not loose any performance. Otherwise this way is not recommended to use on majority of device types because this way all the traffic is handled by device’s (relatively slow) CPU.