Hello,

Has anyone tried or had success with connecting two separate Internet gateways via one physical Ethernet interface? We have basically, two cable modems, plugged into a switch and connected to our MT router's WAN port. One cable modem is the default GW for the router. For the other modem, we have a pre-routing mangle rule to mark the traffic of certain addresses (mark routing). Then we have a second route setup with the routing-mark corresponding to the mangle rule to route that traffic.

This routing method seems to work fine when the multiple gateways are on separate physical WAN interfaces. The second route always shows as invalid with more than one gateway on the same WAN interface though (the second route is using a routing-mark). Can anyone confirm or deny if what we are trying to do should/could work? Can two Internet gateways be used from a single WAN port on the MT router? We are using static routing/addressing (no pppoe, dchp, etc...).

Thanks in advance.

yes, running multiple gateways out a single interface is no problem. You could run reach thousands if you wanted. You just tell the router where to send the next packet and it will be fine.

One problem that you will probably end up with is because of the way the cable networks work. You have a single interface on 2 cable modems. Both cables modems see the same router interface, meaning the same MAC address. If you let source IPs or ARPs accidentally cross the boundaries the cable node will start sending traffic the wrong way because it will learn it has 2 paths to the same MAC address. Please test this theory on your setup and see if you notice that with packet sniffer as well.

_________________
Colo and Wholesale Bandwidth Available! Sales at SanDiegoBroadband dot com

Thanks for the ideas. That's what I would think to - but I must be missing something simple here. These are not traditional cable modems. They are more like commercial DSL modems. They have 4 LAN ports, integrated WIFI and are not MAC address dependant like regular cable modems. Each one also has a public /28 subnet statically routed to it's LAN port (our WAN port). I have made a standard routing config on this MT, just like all the other MTs we have - but this one will not let the second route go active on the same port. This is the first time I have ever tried to run two Internet gateways on one WAN port.

I said earlier that both cable modems were plugged into a single switch and plugged into the WAN port of a single MT for illustrative purposes. In reality, there are 2 cable modems, plugged into 2 Mikrotiks, connected to each other by a wireless backhaul. Each modem/MT combo provides Internet to each of two campuses. Both modems are connected to each MT on a single WAN port. My first description is also technically accurate - just much simpler.

Basically we want one property to be able to fail over to the other if it goes down - or to route certain/extra traffic from one to the other. This has always worked in the past using separate WAN ports (with T-1s, DSLs, etc...). I just can't seem to make it work with 2 WAN gateways on the same WAN port.

The main reason I said all of that is to let you know that both properties are online and routing Internet out their respective cable modems like they should be. This is all pretty basic stuff (routing-wise). I'm just wondering, as far as anyone knows, is there some Ethernet/IP limitation I'm running into? or should this work?

I'm trying to keep this as simple as physically possible to avoid unnecessary points of failure. Ideas?

Thanks again!

The only problem I've ever seen with a similar configuration to yours was when the client used Zyxel ADSL modems. With one turned off, the other worked fine and vice-versa. Try running two on the same Ethernet port and the whole thing ground to a halt.

It turned out that the Zyxels have some sort of stealth IP address they loop back to. It's not documented anywhere, but every modem has exactly the same address. The consequence is that if you put two of them on a single LAN segment (e.g. connect them both to the same switch), the Zyxels clash and everything dies.

With the Zyxel, the answer is to take the tick out of the "Any IP" box on the LAN configuration page. Again, this doesn't appear to be documented anywhere, but as soon as you do it, the problem goes away. Older versions didn't have the tickbox on the GUI and there's a command you have to type at the CLI which does the same thing. I've just looked back through my notes, but I don't appear to have kept that bit of information.

Hopefully your modems are Zyxels or units with a similar problem and the solution above works for you!

Nick.

_________________
N.

Vitell - Asterisk, Linux and network consultants
Unofficial IRC channel: #routerboard on irc.z.je (IPv4) or 6.irc.z.je (IPv6)

i had a setup similar to this on my original MT 532.

My switch supported VLANs, so I had a VLAN for each ISP/router on the outside so that they both weren't connecting to the same "LAN" on the single MT port.

Since both that cablemodem and DSL modem that I had at the time used DHCP for me to get an address from the provider, the DHCP broadcasts went to both providers, and I couldn't discern which was which. The VLAN'd switch help me alleviate that problem.

look guys, if its standard routing then there is something simple going on here. paste an '/ip route print detail' so we can see how its configured.

My guess is that you have 2 default gateways, and the second one is not active because the first one is. This is normal. If you want both active at the same time you can use ECMP (x.x.x.x,y.y.y.y on your gateway). Report back. . .

_________________
Colo and Wholesale Bandwidth Available! Sales at SanDiegoBroadband dot com

i just re-read your original post. your second modem is in another routing table, and thats the one that is not becoming active? enter an /ip route rule to make them active:

/ip route rule
add action=lookup comment="" disabled=no routing-mark=bogons table=bogons

also, make sure you enter in your local subnets into the alternate routing table.

_________________
Colo and Wholesale Bandwidth Available! Sales at SanDiegoBroadband dot com

I really do appreciate the offers for help. I don't really feel right about posting the entire routing table (as is) in a public place though. Then again, omitting info from it may mask the problem. The router has been at the current location for years and has seen at least 6 different gateways and lots of static customers. The routing table is full of public IPs, customer names, carrier names, comments, etc... Not to mention that there are multiple admins working on it - so it's kind of messy. I guess I can just post the pertinent parts and mask the sensitive info - I'm just afraid that may also mask the problem I'm trying to solve here...

Basically, I have two IPs on 1 WAN interface, lets say:
/ip address add address=10.10.0.254/24 interface="WAN1 - Cable"
/ip address add address=10.11.0.254/24 interface="WAN1 - Cable"

a masq:
/ip firewall nat add chain=srcnat action=masquerade out-interface="WAN1 - Cable"


a mangle for the second gateway:
/ip firewall mangle add src-address=172.16.0.122 action=mark-routing new-routing-mark=OTHER_CABLE_OUT chain=prerouting passthrough=yes


and two main routes:
/ip route add dst-address=0.0.0.0/0 gateway=10.10.0.1 distance=0 check-gateway=ping
/ip route add dst-address=0.0.0.0/0 gateway=10.11.0.1 distance=0 check-gateway=ping routing-mark=OTHER_CABLE_OUT


/ip route print detail
32 A S dst-address=0.0.0.0/0 gateway=10.10.0.1 check-gateway=ping interface=WAN1 - Cable gateway-state=reachable
distance=0 scope=255 target-scope=10
38 S ;;; Cable2 Spill-Over
dst-address=0.0.0.0/0 gateway=10.11.0.1 pref-src=10.11.0.254 check-gateway=ping interface=""
gateway-state=unreachable distance=0 scope=255 target-scope=10 routing-mark=OTHER_CABLE_OUT



We have customers setup on 172.16.x.x addresses, among others. Can you see any reason why this basic config wouldn't/shouldn't work? The second route always shows as invalid with an "unknown" interface. I guess I'm going to have to clean this router up eventually - or start over fresh with it. I'm just not sure what or who's service that might "break". This always works perfectly if the routes are on different physical interfaces. I'm almost thinking about trying the VLAN idea dsdee had, just for kicks. So you guys have really run a MT router with multiple gateways on one single WAN interface with success (statically routed - no PPPOE/MLPPP)?

changeip:
I did try an: /ip route rule :
/ip route rule print detail:
Flags: X - disabled, I - inactive
0 src-address=172.16.0.122/32 action=lookup table=OTHER_CABLE_OUT

This didn't seem to make any difference. I'm not sure I completely understand why it's needed though - I've never needed it in the past with multiple WAN ports. Is it related to using a single WAN port?

Also, not sure what you mean when you say "enter in your local subnets into the alternate routing table". Do you mean don't forget my customer natted addresses/subnets on the LAN port??

Thanks again.

turn off check-gateway=ping and see if that helps. I am guessing when the router goes to ping the route, since it's not being route-marked, it just pings that IP from the first gateway and fails. . . so the route is unreachable. You need to probably enter the other important subnets into the alternate routing table so it knows about them. Basically reproduce your connected routes in the alternate table as well. Use the router IP itself as the next hop on those.

_________________
Colo and Wholesale Bandwidth Available! Sales at SanDiegoBroadband dot com

I have tried every combination of options for the route, including disabling/changing ping-check, changing/removing pref-source, etc...

Here's some news though - We just tried to add a static route for a new customer and guess what - it won't work - the route is inactive/invalid no matter what we do. We are running ROS 2.9.51 on this router - I'm kind of afraid to upgrade to 3.x for fear of interrupting service. Everything else is still working properly, including other routes.

Guess I need to fire up the spare/backup router and see if a ROS upgrade helps. Thanks to all for the help - I'll update this thread soon either way. I'm not sure if it will work or not - we may still have a mis-config somewhere. Sometimes an upgrade is the only thing that works though. We'll probably try an upgrade tomorrow morning sometime...

Thanks again!