BGP over L2TP?

RouterOS Forwarding Protocols
1

I want to create a Backup of my backbone over an external Internet connection.
External ISP give me a PPPoE account with a dinamic ip.

So I’ve a RB that do peer with my backbone network and another RB that do PPPoE with other ISP and l2tp tunnel to my datacenter over Internet.
In datacenter there is a RB that have l2tp server enabled and do the peering with the remote device and the core router.

Is possibile to do that?
I’ll try it but the network not work properly.
Sometimes tunnel go down.
Default route is not received

There are some tricks to do it?
network.jpg

2

BGP over L2TP works fine. Probably BGP goes down because installed routes from BGP contains also route to L2TP server in which case it breaks BGP connection.
Do not distribute route to L2TP server over BGP. Also default route is advertised to remote peer only if default-originate is configured.

3

This does not work beause of a bug in the L2TP server:

When a L2TP client attempts to log into the L2TP server, the L2TP server does not always send its reply packets from the correct IP address. The client sees an answer packet coming back, but it carries the wrong source IP, so it discards the packet. The tunnel does not come up if this happens.

The L2TP server actually does not specify a sender IP for its reply packets, so the router uses the IP address of the egress (outgoing) interface as the source IP. That may or may not be correct… and that’s why it works often, but not always - it depends on your routing table / default route whether L2TP works or not. Examples:

Router R1 has a single internet connection, with a default route to it. An external L2TP client with a dynamic IP connects to it. R1 receives the L2TP packets on the same interface as it sends the reply out, the reply packets from the L2TP server happen to have the correct src IP, and L2TP comes up.

However, as soon as more than one internet connection comes into play, it looks different: Example:
Router R1 has two internet connections, ETH1 with IP 123.123.123.10 and ETH2 with IP 55.55.55.20. Lets assume there is a default route via ETH2, i.e. ETH1 is mostly idle. Now an external L2TP client connects through ETH1 to 123.123.123.10… in this case, L2TP does not come up, because R1 sends reply packets via ETH2, and the reply packets end up carrying an incorrect src IP of 55.55.55.20.

once you bring BGP into the picture, i.e. if you use BGP to send traffic through both ETH1 and ETH2 in the above example, you will notice that some L2TP tunnels work, some don’t… depending on whether a particular prefix is reachable through ETH1 or ETH2. actually, if you have two L2TP’s from the client, one going to 123.123.123.10, the other going to 55.55.55.20 you’ll see that always one of them is up, the other is down, reflecting through which ETH the client is reached…

now… you could filter BGP so that you force the reachability of your L2TP client through a specific interface so that L2TP happens to work… but with dynamic IPs on the L2TP client that may require forcing large address ranges to use a specific ISP – which is contrary to the idea of BGP.

the only resort i see would be to use VRFs: the main routing table would contain a static default route to your PPPoE ISP - this is the routing table that the L2TP server uses. BGP would use a separate VRF routing table. that way the L2TP server is not influenced by BGP’s idea of reachability. this should work as long as all your L2TP server traffic should go to one specific ISP. if you want to run L2TP servers on multiple ISP links, you can not do it on a single router because of this bug.

andy
ps: the problem existed on 4.6, and i just re-verified – it exists on 4.10 as well.

4

Mark connections for incoming L2TP packets and set up routing marks or NAT to ensure that reply to specific connection is sent out via correct interface.

5

that does not work.

details of the test:
“input” chain UDP dst port 1701 gets a connection mark i call “L2TP”.
the packet from the L2TP client creates a connection, which is correctly marked L2TP. ok so far.
however the reply from the L2TP server creates a second connection, showing the incorrect source IP.

when i catch the packet in the “output” chain (UDP src port 1701) i can assign a routing mark & i can redirect it to the right interface. but by that time, the packet already has the wrong source ip.
i think the linux kernel looks up the routing table before it sends a packet towards the “output” chain to find a “correct” source IP. that’s why a second connection shows up.

the bug really can not be fixed by mangling.
you need to fix it in the source code – missing bind() call for the UDP socket.
andy

6

It is possible, to force packets with specific src address

[admin@aa] /ip firewall mangle> pr
Flags: X - disabled, I - invalid, D - dynamic
0 chain=postrouting action=mark-connection new-connection-mark=L2TP-CONN
passthrough=yes protocol=udp dst-port=1701

1 chain=postrouting action=mark-packet new-packet-mark=L2TP passthrough=yes
connection-mark=L2TP-CONN

[admin@aa] /ip firewall nat> pr
Flags: X - disabled, I - invalid, D - dynamic
0 chain=srcnat action=src-nat to-addresses=x.x.x.x packet-mark=L2TP

Where x.x.x.x is desired IP address.

7

i tested a fix using NAT during the last week.

first, the mangle/NAT config as given by mrz above does not work: a L2TP client does not necessarily use src port 1701, so in postroute to trigger on dst-port 1701 is probably incorrect. also it seems to me that the connection tracking as given does not lead to the expected results. and anyway, mangling traffic in postroute, on a BGP router… will cause issues for traffic flowing through the router (which may include client L2TP sessions).

but fine… lets try nonetheless. after additional thinking i came up with this, which seemed to make L2TP come up:
/ip firewall nat
add action=src-nat chain=srcnat comment=“l2tp debug” disabled=yes protocol=udp src-port=1701 to-addresses=
xxdesired-sender-ipxx to-ports=1701
the to-ports is necessary because otherwise the UDP reply packet gets translated to port 1024 (contrary to documentation).

the problems then showed up overnight, as the L2TP client on a consumer-grade DSL link was assigned a new IP. L2TP went down but did not come back up, stuck in what seemed being a live-lock: the L2TP client negotiated using a new tunnel ID while the L2TP server was still negotiating using the previous tunnel ID… for hours. i changed the L2TP server settings to allow more than one concurrent login, which cause L2TP to come up, but i’d wonder what happens in this case to statically assigned IPs or OSPF per-interface settings. btw, cisco L2TP clients can hold off new connection attempts, probably exactly for this issue. then… when one day later then the L2TP was again down, when clearing Connections did also not help, i gave up. (when experimenting with NAT/firewalling etc, make sure to frequently clear Connections created by previous rules)

mrz, NAT is not going to be a solution: on a bgp router, having to mangle traffic that flows through the box is problematic, then having to abuse NAT for this fix is even more problematic. and it does not work.

how about setting the L2TP server address explicitely, or per-server interface?
which linux L2TP server is on the box??
andy

8

Confirm the presence of problems using l2tp tunnel for BGP backup.

I resolved with EoIP and DynDNS script but it would have been cleaner with a tunnel l2tp :frowning: