Policy Routing - L2TP and multiple WANs

RouterOS General
1

I’ve been familiar with policy routing for years and understand it pretty well. I’ve spent 2 hours today banging my head trying to figure out if I am just dumb or if something isn’t working like it should.

I have 2 routers on the internet. One is L2TP server, the other is client. The server has multiple WANs, so I’ve setup connection mark, packet mark, and route marks. This works perfect for everything, except L2TP to the router itself. I am seeing the reply’s to the l2tp tunnel setup leaving the wrong interface with the wrong IP, and even weirder is that I don’t even see a trace of it in the connection-tracking table. Has anyone had this same issue?

The picture below shows the client making the connection thru 3-development (2nd wan), and the reply packets immediately get sent back via default gateway and the wrong IP address.

Here is the mangling setup, which works perfect for anything else:

I’m also posting the routing table that is used for the route mark because i’m hoping this is where my problem is.

I can see the connection mark counter increase so I believe the packets are getting marked, however I cannot find them ANYWHERE in connection tracking, even when using CLI to search. I also see the packet marks and route marking counters increasing. SSH connections taking the exact same path works, so it must be something to do with the L2TP server, the fact it’s UDP, and maybe that the kernel is handling it at a higher level? Please someone make me stop banging my head : )

Sam

L2tp on Multiple IP
Inconsistent L2TP/IPSEC success with Android 4.3 [RESOLVED?]
[Solved] OSPF + Multiple Gateways
ROS6 problem l2tp server witch multiple IPs (alias) on two interface
L2TP two providers
2

have you solved it?

maybe post your mangle rules, not just screenshot w/o information? =)

3

I haven’t solved it, and I am wondering if this is one of those things that you just can’t policy route. Kind of like you can’t block DHCP requests because the kernel gets it first anyhow. ICMP to the router works fine, just not L2TP.

/ip firewall mangle
add action=mark-connection chain=prerouting comment="In via Cox" disabled=no 
    in-interface=1-coxBiz new-connection-mark=in-cox-conn passthrough=yes
add action=mark-packet chain=prerouting comment="" connection-mark=\
    in-cox-conn disabled=no new-packet-mark=in-cox-packet passthrough=yes
add action=mark-packet chain=output comment="" connection-mark=in-cox-conn \
    disabled=no new-packet-mark=in-cox-packet passthrough=yes
add action=mark-routing chain=prerouting comment="" disabled=no \
    new-routing-mark=out-cox packet-mark=in-cox-packet passthrough=yes
add action=mark-routing chain=output comment="" disabled=no new-routing-mark=
    out-cox packet-mark=in-cox-packet passthrough=yes
add action=mark-connection chain=prerouting comment="In via Wireless" \
    disabled=no in-interface=3-development new-connection-mark=\
    in-wireless-conn passthrough=yes
add action=mark-packet chain=prerouting comment="" connection-mark=\
    in-wireless-conn disabled=no new-packet-mark=in-wireless-packet \
    passthrough=yes
add action=mark-packet chain=output comment="" connection-mark=\
    in-wireless-conn disabled=no new-packet-mark=in-wireless-packet \
    passthrough=yes
add action=mark-routing chain=prerouting comment="" disabled=no \
    new-routing-mark=out-wireless packet-mark=in-wireless-packet passthrough=
    yes
add action=mark-routing chain=output comment="" disabled=no new-routing-mark=
    out-wireless packet-mark=in-wireless-packet passthrough=yes
add action=mark-connection chain=prerouting comment="PIP Tunnel" disabled=no 
    in-interface=l2tp-pip new-connection-mark=in-pip-conn passthrough=yes
add action=mark-packet chain=prerouting comment="" connection-mark=\
    in-pip-conn disabled=no new-packet-mark=in-pip-packet passthrough=yes
add action=mark-packet chain=output comment="" connection-mark=in-pip-conn \
    disabled=no new-packet-mark=in-pip-packet passthrough=yes
add action=mark-routing chain=prerouting comment="" disabled=no \
    new-routing-mark=out-pip packet-mark=in-pip-packet passthrough=yes
add action=mark-routing chain=output comment="" disabled=no new-routing-mark=
    out-pip packet-mark=in-pip-packet passthrough=yes




/ip route rule
add action=lookup comment="" disabled=no routing-mark=bogons table=bogons
add action=lookup comment="" disabled=no routing-mark=out-wireless table=out-wireless
add action=lookup comment="" disabled=no routing-mark=out-cox table=out-cox




/ip route
add check-gateway=arp comment="" disabled=no distance=1 dst-address=0.0.0.0/0 gateway=68.15.4.33 routing-mark=out-cox scope=255 target-scope=10
add comment="" disabled=no distance=1 dst-address=10.40.1.0/24 gateway=0-inside routing-mark=out-cox scope=30 target-scope=10
add comment="" disabled=no distance=1 dst-address=10.40.4.0/24 gateway=4-dmz routing-mark=out-cox scope=30 target-scope=10
add comment="" disabled=no distance=1 dst-address=68.15.4.32/27 gateway=1-coxBiz routing-mark=out-cox scope=30 target-scope=10
add check-gateway=arp comment="" disabled=no distance=1 dst-address=0.0.0.0/0 gateway=204.16.169.129 pref-src=204.16.169.130 routing-mark=out-wireless scope=255 \
    target-scope=10
add check-gateway=arp comment="" disabled=no distance=1 dst-address=8.25.xx.0/24 gateway=204.x6.xxx.129 routing-mark=out-wireless scope=30 target-scope=10
add comment="" disabled=no distance=1 dst-address=10.0.0.100/30 gateway=3-development routing-mark=out-wireless scope=30 target-scope=10
add comment="" disabled=no distance=1 dst-address=10.x.1.0/24 gateway=0-inside routing-mark=out-wireless scope=30 target-scope=10
add comment="" disabled=no distance=1 dst-address=10.x.4.0/24 gateway=4-dmz routing-mark=out-wireless scope=30 target-scope=10
add check-gateway=arp comment="" disabled=no distance=1 dst-address=204.x6.xxx.0/24 gateway=204.x6.xxx.129 routing-mark=out-wireless scope=30 target-scope=10
add comment="" disabled=no distance=1 dst-address=0.0.0.0/0 gateway=10.9x.0.1 routing-mark=out-pip scope=255 target-scope=10
add comment="" disabled=no distance=1 dst-address=10.22.0.0/16 gateway=l2tp-amistad routing-mark=out-pip scope=255 target-scope=10
add comment="" disabled=no distance=1 dst-address=10.33.0.0/16 gateway=l2tp-delmar routing-mark=out-pip scope=255 target-scope=10
add comment="" disabled=no distance=1 dst-address=10.44.1.0/24 gateway=0-inside routing-mark=out-pip scope=255 target-scope=10
add comment="" disabled=no distance=1 dst-address=10.44.4.0/24 gateway=4-dmz routing-mark=out-pip scope=255 target-scope=10
add comment="" disabled=no distance=1 dst-address=10.9x.0.1/32 gateway=l2tp-pip routing-mark=out-pip scope=255 target-scope=10

It’s almost like the DHCP server replies using a completely new connection, and therefore isn’t in the NAT table at the time or something… It would sure be nice if the L2TP server would just automatically reply with the same IP endpoint that the request came in on.

4

i can confirm:
the bug is in the L2TP server. it sends its packets without specifying a sender IP.
the packets then get routed, and end up on an egress interface… and HERE the source IP gets assigned.
what you see on the wire is the IP of the egress interface.

of course this is wrong. an outside L2TP client can send a request to a destination IP, but gets replies coming back from a different IP. this makes L2TP incompatible with having multiple WAN connections.

it is actually a violation of RFC1122, 3.3.4.2 Multihoming Requirements:
“(1) If the datagram is sent in response to a received datagram, the source address for the response SHOULD be the specific-destination address of the request.”

i went through a lengthy exchange with mikrotik support (Janis, ticket 2010030966000498, for V4.6): first they claimed that policy routing is required to support two WAN connections, but actually it seemed that they never really read the emails and the proof i had. in the end it was suggested that this “feature” might work in a 5.0 beta… after all usually the bug is a missing bind() call on the UDP socket of the server to set its sender IP, it is not something tricky or uncommon (especially on a router…)

i currently run L2TP from two MT routers, one on each WAN. i have to permanently dance around this issue, it dictates the whole network structure in other places as well.
actually… i can anyway not yet use both WAN connections, because MT BGP does not handle a full BGP table… another bug, but i read here in the forum they deny there is an issue. “it works” – but probably only if the leased lines are slow. in my case, with two 100Mbps WAN connections, enabling BGP drowns the router (x86 Xeon, lots of RAM) - WinBox disconnects. only with shaping BGP traffic to 2Mbps the router stays responsive. above 70’000 prefixes there seems to be a problem as well. no SNMP, no NAT.

5

A few things.

  1. I am seeing the L2TP problem on v4.13 as well. Basically it is what you descibe, you can winbox using policy based routing into that IP on the secondary routing table, but not L2TP. I have not been able to try v5rc3 yet, but will if the customer gives permission. So it may be fixed in v5, but don’t know yet, also sent information to MT as well.

  2. Full BGP tables don’t have any issue. I have many routers out the net with full tables. As far as having Xeon and gigs of ram, MT don’t use more than 2 gig of RAM so that don’t matter. and Just cause you have a xeon don’t mean its the right stuff for the job as well.

6

I have the same issue.

Mikrotik 5.4 does not seem to have this issue fixed.
And it’s a year after this topic…

What can I say… Mikrotik might be one of the best routing OSes out there, but it has the stupidest bugs and you can’t even get a fix on them… lame.

7

I have also run into this problem. Running v5.7.

Does anyone have any advice on how to work around this problem until such time as it is resolved?

8

As far as I can tell, this isn’t a bug in L2TP, but rather a problem with how your rules are written. You can test this by trying to SSH, PPTP, or WinBox into MikroTik on either wan IP. You will most likely find that one IP works, the other does not.

When a packet goes to leave, even if it knows to use ISP2’s IP, if the routing rules tell it to go out ISP1, it will head that way.
If you look at the Layer3 part of the Packet Flow Diagram, you will see that Local Process OUT goes directly into a Routing Decision, without passing through prerouting, so all your mangle rules and packet marks never touch it. Nowhere in your route rules do you tell it that ISP1 source IP always goes out ISP1, and ISP2 source IP always goes out ISP2.

I have been successfully running dual WAN with L2TP on both interfaces since 5.4 (just after I started on PCC). I posted my config for doing so over in this thread:
http://forum.mikrotik.com/t/l2tp-tunnels-with-multiple-internet-connections-issues/49379/1

I hope it helps.

9

Yeah. We have 6.0rc14 and problem still persist. This is especially visible when you use L2TP/IPsec and try to connect from LAN side to external WAN IP address.

So Mikrotik shame on you for such long lasting and easy to fix bug.

10

Its a known issue with L2TP.

Ticket#2013020866000414.
“We are aware of this L2TP issue. It is reported to developers and will be fixed in the future according to priorities.”

How to fix it:

/ip firewall nat
add action=dst-nat chain=dstnat comment="Fix for an L2TP src-address bug" dst-address="Address you want your L2TP client to connect to" dst-port=1701 protocol=udp to-addresses="src-address that L2TP sends wrong packets with"
11

EDIT:
Merged with previous post.

12

+1
I wait solutions.

13

+1 STILL waiting for a solution.

I don’t know if it would make sense to open a new ticket, but the problem persisits on RouterOS 6.12.

14

The problem was fixed on 6.13 version. Thank you!!!

15

You say fixed in 6.13 but I have a similar problem in 2020 with 6.44.5
Immediately when adding a second wan IP. even with no route on that interface I start getting errors on my l2tp mt to mt vpn.

feb/03 16:30:33 ipsec,info respond new phase 1 (Identity Protection): new.ip.address[500]<=>established.client.ip[500]
feb/03 16:30:33 ipsec,error established.client.ip[ parsing packet failed, possible cause: wrong password
feb/03 16:30:41 ipsec,error established.client.ip[ parsing packet failed, possible cause: wrong password
feb/03 16:31:30 ipsec,info respond new phase 1 (Identity Protection): new.ip.address[500]<=>established.client.ip[[500]
feb/03 16:31:30 ipsec,error established.client.ip[ parsing packet failed, possible cause: wrong password
feb/03 16:31:33 ipsec,error phase1 negotiation failed due to time up new.ip.address[[500]<=>established.client.ip[500] 6ea2c2cb849a1a03:e6932771a8929d28