Community discussions

 
changeip
Forum Guru
Forum Guru
Topic Author
Posts: 3803
Joined: Fri May 28, 2004 5:22 pm

Policy Routing - L2TP and multiple WANs

Fri Apr 09, 2010 12:31 am

I've been familiar with policy routing for years and understand it pretty well. I've spent 2 hours today banging my head trying to figure out if I am just dumb or if something isn't working like it should.

I have 2 routers on the internet. One is L2TP server, the other is client. The server has multiple WANs, so I've setup connection mark, packet mark, and route marks. This works perfect for everything, except L2TP to the router itself. I am seeing the reply's to the l2tp tunnel setup leaving the wrong interface with the wrong IP, and even weirder is that I don't even see a trace of it in the connection-tracking table. Has anyone had this same issue?

The picture below shows the client making the connection thru 3-development (2nd wan), and the reply packets immediately get sent back via default gateway and the wrong IP address.

Image

Here is the mangling setup, which works perfect for anything else:

Image

I'm also posting the routing table that is used for the route mark because i'm hoping this is where my problem is.

Image

I can see the connection mark counter increase so I believe the packets are getting marked, however I cannot find them ANYWHERE in connection tracking, even when using CLI to search. I also see the packet marks and route marking counters increasing. SSH connections taking the exact same path works, so it must be something to do with the L2TP server, the fact it's UDP, and maybe that the kernel is handling it at a higher level? Please someone make me stop banging my head : )

Sam
Colo and Wholesale Bandwidth Available! Sales at SanDiegoBroadband dot com
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8305
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: Policy Routing - L2TP and multiple WANs

Fri Apr 16, 2010 6:36 pm

have you solved it?

maybe post your mangle rules, not just screenshot w/o information? =)
Russian-speaking forum: https://forum.mikrotik.by/. Welcome!

For every complex problem, there is a solution that is simple, neat, and wrong.

MikroTik. Your life. Your routing.
 
changeip
Forum Guru
Forum Guru
Topic Author
Posts: 3803
Joined: Fri May 28, 2004 5:22 pm

Re: Policy Routing - L2TP and multiple WANs

Fri Apr 16, 2010 7:25 pm

I haven't solved it, and I am wondering if this is one of those things that you just can't policy route. Kind of like you can't block DHCP requests because the kernel gets it first anyhow. ICMP to the router works fine, just not L2TP.
/ip firewall mangle
add action=mark-connection chain=prerouting comment="In via Cox" disabled=no 
    in-interface=1-coxBiz new-connection-mark=in-cox-conn passthrough=yes
add action=mark-packet chain=prerouting comment="" connection-mark=\
    in-cox-conn disabled=no new-packet-mark=in-cox-packet passthrough=yes
add action=mark-packet chain=output comment="" connection-mark=in-cox-conn \
    disabled=no new-packet-mark=in-cox-packet passthrough=yes
add action=mark-routing chain=prerouting comment="" disabled=no \
    new-routing-mark=out-cox packet-mark=in-cox-packet passthrough=yes
add action=mark-routing chain=output comment="" disabled=no new-routing-mark=
    out-cox packet-mark=in-cox-packet passthrough=yes
add action=mark-connection chain=prerouting comment="In via Wireless" \
    disabled=no in-interface=3-development new-connection-mark=\
    in-wireless-conn passthrough=yes
add action=mark-packet chain=prerouting comment="" connection-mark=\
    in-wireless-conn disabled=no new-packet-mark=in-wireless-packet \
    passthrough=yes
add action=mark-packet chain=output comment="" connection-mark=\
    in-wireless-conn disabled=no new-packet-mark=in-wireless-packet \
    passthrough=yes
add action=mark-routing chain=prerouting comment="" disabled=no \
    new-routing-mark=out-wireless packet-mark=in-wireless-packet passthrough=
    yes
add action=mark-routing chain=output comment="" disabled=no new-routing-mark=
    out-wireless packet-mark=in-wireless-packet passthrough=yes
add action=mark-connection chain=prerouting comment="PIP Tunnel" disabled=no 
    in-interface=l2tp-pip new-connection-mark=in-pip-conn passthrough=yes
add action=mark-packet chain=prerouting comment="" connection-mark=\
    in-pip-conn disabled=no new-packet-mark=in-pip-packet passthrough=yes
add action=mark-packet chain=output comment="" connection-mark=in-pip-conn \
    disabled=no new-packet-mark=in-pip-packet passthrough=yes
add action=mark-routing chain=prerouting comment="" disabled=no \
    new-routing-mark=out-pip packet-mark=in-pip-packet passthrough=yes
add action=mark-routing chain=output comment="" disabled=no new-routing-mark=
    out-pip packet-mark=in-pip-packet passthrough=yes
/ip route rule
add action=lookup comment="" disabled=no routing-mark=bogons table=bogons
add action=lookup comment="" disabled=no routing-mark=out-wireless table=out-wireless
add action=lookup comment="" disabled=no routing-mark=out-cox table=out-cox
/ip route
add check-gateway=arp comment="" disabled=no distance=1 dst-address=0.0.0.0/0 gateway=68.15.4.33 routing-mark=out-cox scope=255 target-scope=10
add comment="" disabled=no distance=1 dst-address=10.40.1.0/24 gateway=0-inside routing-mark=out-cox scope=30 target-scope=10
add comment="" disabled=no distance=1 dst-address=10.40.4.0/24 gateway=4-dmz routing-mark=out-cox scope=30 target-scope=10
add comment="" disabled=no distance=1 dst-address=68.15.4.32/27 gateway=1-coxBiz routing-mark=out-cox scope=30 target-scope=10
add check-gateway=arp comment="" disabled=no distance=1 dst-address=0.0.0.0/0 gateway=204.16.169.129 pref-src=204.16.169.130 routing-mark=out-wireless scope=255 \
    target-scope=10
add check-gateway=arp comment="" disabled=no distance=1 dst-address=8.25.xx.0/24 gateway=204.x6.xxx.129 routing-mark=out-wireless scope=30 target-scope=10
add comment="" disabled=no distance=1 dst-address=10.0.0.100/30 gateway=3-development routing-mark=out-wireless scope=30 target-scope=10
add comment="" disabled=no distance=1 dst-address=10.x.1.0/24 gateway=0-inside routing-mark=out-wireless scope=30 target-scope=10
add comment="" disabled=no distance=1 dst-address=10.x.4.0/24 gateway=4-dmz routing-mark=out-wireless scope=30 target-scope=10
add check-gateway=arp comment="" disabled=no distance=1 dst-address=204.x6.xxx.0/24 gateway=204.x6.xxx.129 routing-mark=out-wireless scope=30 target-scope=10
add comment="" disabled=no distance=1 dst-address=0.0.0.0/0 gateway=10.9x.0.1 routing-mark=out-pip scope=255 target-scope=10
add comment="" disabled=no distance=1 dst-address=10.22.0.0/16 gateway=l2tp-amistad routing-mark=out-pip scope=255 target-scope=10
add comment="" disabled=no distance=1 dst-address=10.33.0.0/16 gateway=l2tp-delmar routing-mark=out-pip scope=255 target-scope=10
add comment="" disabled=no distance=1 dst-address=10.44.1.0/24 gateway=0-inside routing-mark=out-pip scope=255 target-scope=10
add comment="" disabled=no distance=1 dst-address=10.44.4.0/24 gateway=4-dmz routing-mark=out-pip scope=255 target-scope=10
add comment="" disabled=no distance=1 dst-address=10.9x.0.1/32 gateway=l2tp-pip routing-mark=out-pip scope=255 target-scope=10
It's almost like the DHCP server replies using a completely new connection, and therefore isn't in the NAT table at the time or something... It would sure be nice if the L2TP server would just automatically reply with the same IP endpoint that the request came in on.
Colo and Wholesale Bandwidth Available! Sales at SanDiegoBroadband dot com
 
azg
Frequent Visitor
Frequent Visitor
Posts: 50
Joined: Thu Jun 17, 2010 1:40 pm

Re: Policy Routing - L2TP and multiple WANs

Thu Jun 17, 2010 3:06 pm

i can confirm:
the bug is in the L2TP server. it sends its packets _without specifying a sender IP_.
the packets then get routed, and end up on an egress interface... and HERE the source IP gets assigned.
what you see on the wire is the IP of the egress interface.

of course this is wrong. an outside L2TP client can send a request to a destination IP, but gets replies coming back from a different IP. this makes L2TP incompatible with having multiple WAN connections.

it is actually a violation of RFC1122, 3.3.4.2 Multihoming Requirements:
"(1) If the datagram is sent in response to a received datagram, the source address for the response SHOULD be the specific-destination address of the request."

i went through a lengthy exchange with mikrotik support (Janis, ticket 2010030966000498, for V4.6): first they claimed that policy routing is required to support two WAN connections, but actually it seemed that they never really read the emails and the proof i had. in the end it was suggested that this "feature" might work in a 5.0 beta... after all usually the bug is a missing bind() call on the UDP socket of the server to set its sender IP, it is not something tricky or uncommon (especially on a router...)

i currently run L2TP from two MT routers, one on each WAN. i have to permanently dance around this issue, it dictates the whole network structure in other places as well.
actually... i can anyway not yet use both WAN connections, because MT BGP does not handle a full BGP table... another bug, but i read here in the forum they deny there is an issue. "it works" -- but probably only if the leased lines are slow. in my case, with two 100Mbps WAN connections, enabling BGP drowns the router (x86 Xeon, lots of RAM) - WinBox disconnects. only with shaping BGP traffic to 2Mbps the router stays responsive. above 70'000 prefixes there seems to be a problem as well. no SNMP, no NAT.
 
gmsmstr
Trainer
Trainer
Posts: 940
Joined: Fri Jun 04, 2004 2:22 am
Location: St. Louis, MO
Contact:

Re: Policy Routing - L2TP and multiple WANs

Fri Nov 19, 2010 3:08 am

A few things.

1. I am seeing the L2TP problem on v4.13 as well. Basically it is what you descibe, you can winbox using policy based routing into that IP on the secondary routing table, but not L2TP. I have not been able to try v5rc3 yet, but will if the customer gives permission. So it may be fixed in v5, but don't know yet, also sent information to MT as well.

2. Full BGP tables don't have any issue. I have many routers out the net with full tables. As far as having Xeon and gigs of ram, MT don't use more than 2 gig of RAM so that don't matter. and Just cause you have a xeon don't mean its the right stuff for the job as well.
Dennis Burgess, MCTCE, MTCNA, MCTCTE, MTCWE, MTCNIE, A+, N+, MCP, MTCSE Mikrotik Certified Consultant / Trainer
Need Mikrotik Support: http://www.linktechs.net -- Link Technologies, Inc.
-- Author of "Learn RouterOS: Second Edition"
 
User avatar
Cha0s
Forum Veteran
Forum Veteran
Posts: 890
Joined: Tue Oct 11, 2005 4:53 pm

Re: Policy Routing - L2TP and multiple WANs

Mon May 30, 2011 6:14 pm

i can confirm:
the bug is in the L2TP server. it sends its packets _without specifying a sender IP_.
the packets then get routed, and end up on an egress interface... and HERE the source IP gets assigned.
what you see on the wire is the IP of the egress interface.

of course this is wrong. an outside L2TP client can send a request to a destination IP, but gets replies coming back from a different IP. this makes L2TP incompatible with having multiple WAN connections.

it is actually a violation of RFC1122, 3.3.4.2 Multihoming Requirements:
"(1) If the datagram is sent in response to a received datagram, the source address for the response SHOULD be the specific-destination address of the request."

i went through a lengthy exchange with mikrotik support (Janis, ticket 2010030966000498, for V4.6): first they claimed that policy routing is required to support two WAN connections, but actually it seemed that they never really read the emails and the proof i had. in the end it was suggested that this "feature" might work in a 5.0 beta... after all usually the bug is a missing bind() call on the UDP socket of the server to set its sender IP, it is not something tricky or uncommon (especially on a router...)

i currently run L2TP from two MT routers, one on each WAN. i have to permanently dance around this issue, it dictates the whole network structure in other places as well.
actually... i can anyway not yet use both WAN connections, because MT BGP does not handle a full BGP table... another bug, but i read here in the forum they deny there is an issue. "it works" -- but probably only if the leased lines are slow. in my case, with two 100Mbps WAN connections, enabling BGP drowns the router (x86 Xeon, lots of RAM) - WinBox disconnects. only with shaping BGP traffic to 2Mbps the router stays responsive. above 70'000 prefixes there seems to be a problem as well. no SNMP, no NAT.
I have the same issue.

Mikrotik 5.4 does not seem to have this issue fixed.
And it's a year after this topic...

What can I say... Mikrotik might be one of the best routing OSes out there, but it has the stupidest bugs and you can't even get a fix on them.... lame.
 
samct
just joined
Posts: 11
Joined: Fri Nov 04, 2011 3:58 pm

Re: Policy Routing - L2TP and multiple WANs

Fri Nov 04, 2011 4:01 pm

I have also run into this problem. Running v5.7.

Does anyone have any advice on how to work around this problem until such time as it is resolved?
 
CCDKP
Member Candidate
Member Candidate
Posts: 170
Joined: Fri Jan 28, 2011 11:24 pm
Location: Midwest, United States

Re: Policy Routing - L2TP and multiple WANs

Wed Nov 09, 2011 5:09 pm

I have also run into this problem. Running v5.7.
Does anyone have any advice on how to work around this problem until such time as it is resolved?
As far as I can tell, this isn't a bug in L2TP, but rather a problem with how your rules are written. You can test this by trying to SSH, PPTP, or WinBox into MikroTik on either wan IP. You will most likely find that one IP works, the other does not.

When a packet goes to leave, even if it knows to use ISP2's IP, if the routing rules tell it to go out ISP1, it will head that way.
If you look at the Layer3 part of the Packet Flow Diagram, you will see that Local Process OUT goes directly into a Routing Decision, without passing through prerouting, so all your mangle rules and packet marks never touch it. Nowhere in your route rules do you tell it that ISP1 source IP always goes out ISP1, and ISP2 source IP always goes out ISP2.
Image

I have been successfully running dual WAN with L2TP on both interfaces since 5.4 (just after I started on PCC). I posted my config for doing so over in this thread:
http://forum.mikrotik.com/viewtopic.php?f=2&t=54228

I hope it helps.
CC_DKP: MTCNA, MTCRE, MTCWE, MTCTCE, part-time packet wrangler
 
ayufan
Member
Member
Posts: 331
Joined: Sun Jun 03, 2007 9:35 pm
Contact:

Re: Policy Routing - L2TP and multiple WANs

Wed May 15, 2013 2:22 pm

Yeah. We have 6.0rc14 and problem still persist. This is especially visible when you use L2TP/IPsec and try to connect from LAN side to external WAN IP address.

So Mikrotik shame on you for such long lasting and easy to fix bug.
hAP AC, TP-Link Archer C7 v2, RB951G, RB450G, RPI2, RPI zero
 
User avatar
tomaskir
Trainer
Trainer
Posts: 1110
Joined: Sat Sep 24, 2011 2:32 pm
Location: Slovakia

Re: Policy Routing - L2TP and multiple WANs

Sat May 18, 2013 11:36 am

Its a known issue with L2TP.
Ticket#2013020866000414.
"We are aware of this L2TP issue. It is reported to developers and will be fixed in the future according to priorities."
How to fix it:
/ip firewall nat
add action=dst-nat chain=dstnat comment="Fix for an L2TP src-address bug" dst-address="Address you want your L2TP client to connect to" dst-port=1701 protocol=udp to-addresses="src-address that L2TP sends wrong packets with"
Last edited by tomaskir on Sat May 18, 2013 4:29 pm, edited 2 times in total.
Unimus - configuration management, automation and backup solution
Mass Config Push, network-wide RouterOS upgrades, and more!
 
User avatar
tomaskir
Trainer
Trainer
Posts: 1110
Joined: Sat Sep 24, 2011 2:32 pm
Location: Slovakia

Re: Policy Routing - L2TP and multiple WANs

Sat May 18, 2013 4:19 pm

EDIT:
Merged with previous post.
Unimus - configuration management, automation and backup solution
Mass Config Push, network-wide RouterOS upgrades, and more!
 
vovanfz
just joined
Posts: 11
Joined: Thu Aug 22, 2013 8:37 am

Re: Policy Routing - L2TP and multiple WANs

Mon Apr 28, 2014 9:28 am

+1
I wait solutions.
 
Jivo
newbie
Posts: 34
Joined: Wed Jun 08, 2011 11:44 pm

Re: Policy Routing - L2TP and multiple WANs

Sun May 11, 2014 4:51 pm

+1 STILL waiting for a solution.

I don't know if it would make sense to open a new ticket, but the problem persisits on RouterOS 6.12.
 
vovanfz
just joined
Posts: 11
Joined: Thu Aug 22, 2013 8:37 am

Re: Policy Routing - L2TP and multiple WANs

Mon May 19, 2014 9:55 am

The problem was fixed on 6.13 version. Thank you!!!

Who is online

Users browsing this forum: Bing [Bot] and 38 guests