Try the simplest one (no need of added routing tables, everything stays in "main", no need of mangle rules).
Simpler Failover for two Gateways I found working - #19 by jaclaz
At first glance of what you posted you are not anyway following the original (simpler but not simplest) Filo's approach, in that one you need only one extra routing table and only one mangle rule).
Maybe related, maybe not, the given methods use the "normal", icmp type, likely "simple" would do as well, you seem to be using type=tcp-conn instead, I have no idea how that works.
I think it would be easier if you start from scratch as opposed to edit your earlier attempt.