Routed OSPF network 2 WAN DSL connections to ISP

RouterOS Forwarding Protocols
1

I have 2 RB750’s connected together via a 5ghz link, this is setup with WDS layer 2 transparent. I have then created a VLAN between the routers and followed the wiki guide on setting up OSPF which works fine. Both routers are connected to the same ISP via DSL lines using PPPOE into the TIK’s. There is no NAT everything is routed.
The ISP has allocated 2 x /29 ranges for each connection. What I hope to achieve is failover, so that if the DSL connection on Router TIK1 fails, it will use the default route (distributed by OSPF) via TIK2 to the internet. My ISP has told me that I can egress either /29 IP range on either connection, and for incoming traffic they will route the corresponding /29 to the correct DSL and if it fails, route the incoming to the other connection, OSPF should then sort the routing.

The problem:
Lets say I am connected to TIK1 and I break the local DSL connection. The default gateway for that connection is removed from routing, which is correct, and the OSPF default route become active (via TIK2) but the traffic fails. If I traceroute I can see the packets get as far as TIK2 but then are dropped. The ISP says they never see them hit them, yet if I run a packet sniff on the PPPOE of TIK2 I can see the packets going out.

Any ideas? I’m presuming this should work, as its simple routing, I am using V6.12 on the TIK’s wondering if the new IP security features might be stopping the TIK from sending out a packet with a source address that is not local to it? Or if the packets are seen in the packet sniff pcap on the PPPOE link, then can I assume they are 100% going to the ISP and perhaps they are dropping it?

Any suggestions welcome.

Thanks

W
Drawing1.jpg

2

Where are the IPs from the /29 of the DSL connection on TIK1 configured? On the PPPoE interface? If so, the IPs may go invalid when the connection falls down and be withdrawn from OSPF which would mean that TIK2 would no longer be able to find a route for those IPs.

What are your firewall rules? both routers

How are your IP addresses configured? both routers

What are your routes while everything is online? both routers

What are your routes while one DSL connection is offline? both routers

It mostly sounds like the ISP is not actually accepting traffic from the TIK1 /29 via the DSL connection on TIK2. Can you ping out with one of the TIK1 IPs from TIK2 while both connections are live? If the ISP is accepting egress traffic from both /29s, your return packet should come back in via the connection to TIK1.

It may take 3 minutes for the ISPs BRAS to notice that the PPPoE session for DSL1 is dead if you just pull the phone cord. How long have you waited for the ISPs device to notice that the PPPoE session is failed?

If you disable a PPPoE interface on TIK1, the MikroTik should tear down the PPP connection nicely and your ISP will be aware of the link being down immediately.

Are you using PPPoE on the DSL modem or on the MikroTik? Is the DSL modem acting as a pure bridge? Or is the DSL modem actually your gateway for the /29 on each MikroTIk? I’ve seen some funky setups, especially with 2-wire DSL modem/routers on AT&T.

3

Hi Lambert, thanks for the reply, this is really bugging me, answers below

Where are the IPs from the /29 of the DSL connection on TIK1 configured? On the PPPoE interface? If so, the IPs may go invalid when the connection falls down and be withdrawn from OSPF which would mean that TIK2 would no longer be able to find a route for those IPs.

This is a UK ISP, There is a static IP (a single ip - /32) that the TIK learns from the ISP via PPP (dynamic entry in the TIK on PPPOE, but always the same ip issued), good point though the /29 is built against the PPPOE Interface so will most probably go bad when the PPPOE goes down, It will mark it down, I need to move the /29 to be on an ETH port.
Right now I am focussing on getting a non local IP address (but valid for the other connection) to egress a packet from the router

It mostly sounds like the ISP is not actually accepting traffic from the TIK1 /29 via the DSL connection on TIK2. Can you ping out with one of the TIK1 IPs from TIK2 while both connections are live? If the ISP is accepting egress traffic from both /29s, your return packet should come back in via the connection to TIK1.

For now this is what I have been doing to test, back to real basics, both connections live, ping a target IP outside of my ISP ip ranges (but with another TIK I can use to see if the ping hits) I put a static route to the outside-TIK (outside my ISP), so that if you ping that IP from TIK1 it routes via TIK2. If I run a packet sniff on TIK2 on the PPPOE interface and grab the PCAP I can see the ICMP packet leaving TIK2 with the source address set to the /29 on TIK1, so in other words correct. The ISP run a similar trace on their side, but never see that packet, and my test outside TIK never see the packet.
So either the TIK is not really sending it (maybe something filters it before it actually goes PPP), or it does send it but my ISP drops it, but they are very clued up and running the pcap on their LNS I believe, so effectively the other side of the PPP session.

Its not easy for me to paste the config data, its a live service, so I am going to recreate the scenario with 2 vanilla TIK’s and see if I observe the same problem, if so I will paste the config of those, much simpler routers.

Regards

Wayne

4

Do you have RP filtering turned on?

5

Nope IP Filtering is off (=no), IP forward is on.

6

This is resolved, the ISP had RP filtering enabled