Here is a setup at one of my customer locations:

I have a 6Mbit DSL circuit with static public IP's serving an RB2011.
I also have a 150Mbit Cable circuit with 1 dynamic IP serving an RB750GL.
The 2 routers are connected together with a "backbone" ethernet connection.

I can access a NAS server behind the RB2011 through a dedicated static public IP on the RB2011 with no issues.
(I use 2 rules for srcnat and dstnat to create 1 to 1 NATing.)

But I also have need to access the NAS server through the higher speed 150 Mbit Cable ISP with a port forward.

So when I port forward port example 22222 one the RB750GL to 22 on the NAS server the SYN packet reaches the NAS via the backbone but the SYN,ACK packet is routed back out through the DSL circuit instead of back through the backbone to the Cable router where the connection originated.

Do I need to MARK the packets to make this work? That's what I am thinking but now sure.

You could also give your other connection your static ip, it may work.

I think since you already have port forwarding on your DSL that it would default to that being that the config for it comes first. RouterOS goes through firewall rules linearly from the first rule to the last.

You could also give your other connection your static ip, it may work.

I think since you already have port forwarding on your DSL that it would default to that being that the config for it comes first. RouterOS goes through firewall rules linearly from the first rule to the last.

First thing - Thanks for your response.

That is my thought too. The NAT rules for the 1:1 rules on the DSL are possibly interfering. But temporarily disabling those rules don't change the behavior.

What do you mean by "give your other connection your static ip"?

If you own a public ip address you can assign it to another connection.

It may seem a lot better to migrate everything to your cable and use dsl as failover. You will need to apply more logic to get port forwarding right. You could for example use 2 ports on your NAS so that you can use a different source NAT for each. You can also assign 2 IPs to your NAS via vlan and work through that.

If you own a public ip address you can assign it to another connection.

It may seem a lot better to migrate everything to your cable and use dsl as failover. You will need to apply more logic to get port forwarding right. You could for example use 2 ports on your NAS so that you can use a different source NAT for each. You can also assign 2 IPs to your NAS via vlan and work through that.

Good call. I was thinking about putting the NAS on the cable connection because of the 150 down and 10 up speeds (and unlimited bandwidth in the case of this customer). I am using 2 ports on the NAS but I am still wondering why the routing is mangled (bad choice of words, because if don't mean mangled yet, unless I do some MARKing).

What is the intend? Active and backup router? Then VRRP is your answer.

What is the intend? Active and backup router? Then VRRP is your answer.

You guys are so far ahead of my knowledge.

I just want to have 2 ways to get to the NAS server without having to move it to the other LAN. If I do that then the internal machines on the existing LAN will no longer see the NAS and what makes it critical is that the internal machines are VM's that are all configured to boot their images from the NAS.

So I really have created a monster

Could you perhaps make a diagram with IP addressing on it? It's not clear to begin with to which address the 750GL ports forwards.

I think it should not really be an issue to give your NAS (1) a public routable IP (DSL) and then (2) an RFC 1918 private IP that is masqueraded using the cable's ISP public IP address.
It would require 2 interfaces on the NAS though.

At first I was under the impression you wanted you LAN to have 2 exits (one via cable, one via DSL) in a failover scenario. Sorry, should have read better too:-)

edit: added a sentence:-)

Could you perhaps make a diagram with IP addressing on it?

I think it should not really be an issue to give your NAS (1) a public routable IP (DSL) and then (2) an RFC 1918 private IP that is masqueraded using the cable's ISP public IP address.
It would require 2 interfaces on the NAS though.

At first I was under the impression you wanted you LAN to have 2 exits (one via cable, one via DSL) in a failover scenario. Sorry, should have read better too:-)

I'll do up a quick diagram ASAP.

Here is a hand drawn diagram

I am still thinking that connections/packet marks or routing marks are required but I don't know how to implement them.

so thats your setup. You only need 1 router unless you are expecting a router to fail or have multiple locations. Using only 1 of the routers you can combine both internets the way you want and than do a general src and dst nat.

Is there a reason for having 2 seperate routers?
Is your LAN subnet 255.255.0.0?
There are no packets that are needed to be mark

so thats your setup. You only need 1 router unless you are expecting a router to fail or have multiple locations. Using only 1 of the routers you can combine both internets the way you want and than do a general src and dst nat.

Is there a reason for having 2 seperate routers?
Is your LAN subnet 255.255.0.0?
There are no packets that are needed to be mark

I see your point in only needing one router. It is a bit of a legacy situation and so I never got around to getting the customer's ISP links all on the RB2011.
Each router has an entry in the Firewall -> AddressLists called "LAN" and is 192.168.0.0/16.

So without overhauling, I would like to fix the problem. Here the output on a log server from the firewall "log" packet debug statements...

2014-11-10T12:26:34-05:00 10.1.1.2 firewall,info DEBUG1 forward: in:ether4 out:ether3-lan-private-master, src-mac d4:ca:xx:xx:xx:5d, proto TCP (SYN), 173.xxx.yyy.zzz:54797->192.168.88.195:2222, len 60

2014-11-10T12:26:34-05:00 10.1.1.2 firewall,info DEBUG2 forward: in:ether3-lan-private-master out:pppoe-out1, src-mac 74:d4:xx:xx:xx:06, proto TCP (SYN,ACK), 192.168.88.195:2222->173.xxx.yyy.zzz:54797, len 60

As you can see, the SYN-ACK goes out the DSL (pppoe-out1) interface instead of back through the ether4 interface (the backbone).

Configuring 2 routers for port forwarding of 1 device is actually twice the work.

More to this you need to set your static LAN routes first so that both routers can communicate on LAN. The configuration is basically similar for both routers. Using your DSL as failover simply set the route that uses the DSL as a lower priority or longer length. Than create a general NAT and port forwarding on both routers. You use the routes/gateways to determine which path is taken first. To make it simpler, once the router connects to internet, simply make static the dynamic route and copy it over to the other router.

For the sake of simplicity you can have 1 router operate DHCP and DNS. You can also have both routers running DHCP on the same subnet but just divide the ip pool between them.

Configuring 2 routers for port forwarding of 1 device is actually twice the work.

More to this you need to set your static LAN routes first so that both routers can communicate on LAN. The configuration is basically similar for both routers. Using your DSL as failover simply set the route that uses the DSL as a lower priority or longer length. Than create a general NAT and port forwarding on both routers. You use the routes/gateways to determine which path is taken first. To make it simpler, once the router connects to internet, simply make static the dynamic route and copy it over to the other router.

For the sake of simplicity you can have 1 router operate DHCP and DNS. You can also have both routers running DHCP on the same subnet but just divide the ip pool between them.

Thanks. The static routes are in place. So right now by default, outbound users on the DSL router suffer the slow performance and the users on the Cable router have the luxury of the high performance. Setting a lower priority sound like a good idea. That should give the clients behind the DSL router equal performance to the other guys. As for "making static the dynamic route" - I guess you mean the dynamic Cable route needs to be made static and then copied over to the DSL router?