Just figured it out now because of your reply. Actually I'm just a curious with some RBs running and trying to make things work the correct way and understand why/how they work.
I added:
/ip firewall filter add action=accept chain=forward comment="WireGuard S2S management" connection-state=established,related in interface=wireguard1
/ip firewall filter add action=drop chain=forward in-interface=wireguard1
Worked just as I wanted. Can access subnet on site B, site B can reply, but cant access site A subnet.