I'm going to try my best to describe what I have without making it too complicated.

Basically, I have two wireless links to another WISP who supplies me with my bandwidth. Both links connect to different water towers, which have an additional hop back to their main office where I have a Mikrotik 450G. Ether1 is the WAN connection to their fiber, Ether2 is the feed to water tower A, and Ether 3 is the feed to water tower B. I also have a Mikrotik 450G at my main tower. Ether1 goes to my LAN, Ether2 is the feed from water tower A, and Ether 3 is the feed from water tower B.

The links to each water tower are on their own routed subnet. I have OSPF distributing the routes, and is configured to give priority to water tower A. If I shut off "A", it fails over seamlessly to "B". So all of that is working properly. If both links fail, I have another static route with a high path cost that will kick in if both OSPF routes are lost, and that will route my traffic to a backup DSL connection on my LAN.

All of that works fine, but my issue is that OSPF is only good at determining if things are working between the two 450G routers. If the other WISP has an outage on their fiber (or anything upstream from the 450G on their side, the backup route will never kick in and my network will be offline.

Before setting up OSPF, I had both connections to the other ISP bridged, and I would manually enable one of the bridge ports to avoid creating a loop. I then made a script which would ping an offsite IP address and change the cost of the backup route. The ping to that specific IP was forced through the WISP so it didn't flip flop once it was running on the DSL connection.

That all worked fine until implementing OSPF because I can only seem to get the ping to go out one path reliably. I have tried to change my backup route to use ECMP and simply enter the gateway for both routes to the other WISP, but it doesn't seem to realize when the primary gateway is unreachable, and therefore won't use the second gateway.

Does all of that make sense, and does anyone have any ideas on how to get my ping/script to work properly, or have an even better method that I could be using to route my traffic to the DSL connection in the event that one or both of the OSPF routes are still up but something upstream is broken?

Thanks in advance,
Joe

The simplest method would be to only have the RB450G at the other WISP's location announce a default route if it already has one. Then withdraw it when there's problems.

Set the 'distribute-default' property on the OSPF instance on that router to 'if-installed-as-type-1' so that it distributes a default route into OSPF only if there is already a 0.0.0.0/0 route in its own routing table, and as type 1 so that cost gets added up.

Then add a static default route to the routing table. Then add a Netwatch item that pings an upstream IP to test for connectivity. Have that item disable the default route if the Netwatch test fails, and enable it again when it succeeds. How do you reach the test IP even when the default route is down so it can ever come back up? Add a specific route.

So once the target goes down the static default gets disabled, OSPF on the 450G at the partner WISP location withdraws its default route throughout your entire OSPF domain, and your floating static via your DSL backup link kicks in. When the target comes back up OSPF introduces a default route again, the floating static gets pushed down, and traffic flows via the fiber link.

Here an example that assumes that your IP on the fiber link is 1.1.1.2/30 with a gateway of 1.1.1.1, and that there's only one OSPF instance. Your test target upstream is 2.2.2.2.
Example code written in browser window only and may contain errors, but you get the idea.

Addendum: personally I am not a big fan of hard failed circuits that enable themselves again so I'd generate alerts via my NMS that traffic swung to the DSL line, and then manually establish the fiber link is up again and manually re-enable the static default route, and run the NetWatch task with an empty up event script. Rerouting through a different circuit usually changes your NAT IP (I'm assuming you do not have owned public IPs in your own AS announced via BGP on both circuits), so it breaks existing connections. Flapping that back and forth every five minutes on a semi persistent issue sucks for customers.

Thanks fewi!

I implemented your suggestion of removing the default route on the upstream router and it works great!

I also understand what you are saying about circuits that can cause flapping, but in my case, the DSL connection is slow and would be painful to leave customers on any longer than necessary. I have had to run on it for a full day before when there was a fiber cut, and while things were slow, at least people were online and it kept the phone calls to a minimum. So rather than using netwatch, I am using my script that I had before implementing OSPF as it has a lot more flexibility to prevent flapping, although it is still possible.

You are correct that I don't have my own BGP'd IP space, so if I end up on the DSL connection, everything does get NAT'd. I do get SMS notifications if anything goes down, so in the event of the primary link going up and down a bunch, I can login and force things to the backup until the upstream problem is fixed.

Anyway, thanks again for the suggestions. They worked like a charm!

I'm glad you got it working.

jcremin. hiii . i need help. can you share your code with me?
i need to do ospf with 2 ubiquiti pptp and two.rb450 to make bonding and failover and cost
thankssss