Bypass NAT for zerotier

I have a typical natted router with zerotier behind it. My upstream ISP is also natted and provides a /26 subnet to me. I am currently only using one of those ip addresses assigned via dhcp client.

Zerotier is having connection issues in this setup. The documentation specifically says to avoid double NAT. What I am seeing is pings work fine but stateful connections fail. I am seeing incoming and outgoing upd holepunch traffic on multiple sequential ports with no actual connection established.

I would like to assign another static address to the zerotier. All traffic except for zerotier will be natted and go out the original IP. All zerotier transport traffic will not be natted and instead will be statically using the second IP.

I think this is done with srcnat/dstnat rules placed above the existing masquerade rule. But, I am having trouble distinguishing zerotier transport traffic or mangling a connection mark to the output of the zerotier interface.

Can someone help me with some mangle or nat rules that will bypass nat for zerotier?