You don’t need to enable auto-isolate. You are not specific enough regarding what means “network hangs”, so I can only speculate that there is some incompatibility between RSTP versions so the L2 loop does not get cut and broadcast traffic exhausts all the bandwidth. Hence I would make the physical topology a star or a line as a diagnostic step, enable RSTP on all devices and check the bridge status on all devices where it is possible (I suspect SwOS doesn’t provide bridge state monitoring). Adjusting priorities should show you whether the root bridge moves accordingly and thus find out which devices do not understand each other’s dialect of RSTP.