I've tried the following now:
/ip firewall mangle
add action=mark-packet chain=input comment=\
"Mark incoming packets on ISP2 as ISP2-in" in-interface=ether2 \
new-packet-mark=ISP2-wan-ip passthrough=yes protocol=icmp
add action=mark-connection chain=input comment=\
"Mark incoming connections on ISP2 as ISP2-in" new-connection-mark=\
ISP2 packet-mark=ISP2-wan-ip passthrough=yes
add action=route chain=prerouting comment=\
"Packets with ISP2 mark to ISP2 routing table" connection-mark=\
ISP2 log=yes log-prefix=ISP2 passthrough=yes route-dst=ISP2.gateway.IP
I've tried the last rule using both the "route" and mark-routing (to the routing table with only the default GW for ISP 2), however this doesn't work, as it appears the router doesn't know where to output the packet? The counters do count up, so the rules are hit/executed as expected.
ISP2 prerouting: in:ether2 out:(unknown 0), connection-mark:ISP2 connection-state:established src-mac 3c:5e:c3xxxx, proto ICMP (type 8, code 0), Ext.Src.Ip.Addr->ISP2.ip.addr, len 84
To sum up: I need ICMP replies to be sent out the same interface they ware received from so I can monitor the backup/slave connection in a failed over configuration.