Community discussions

MikroTik App
 
brendanp
just joined
Topic Author
Posts: 20
Joined: Sun Dec 04, 2011 6:07 am

Site to site Layer 2 VPN with full ethernet MTU -- over IPv6

Tue Jul 13, 2021 2:39 pm

Hello!

I'm trying to set up a site to site VPN between two Mikrotik ROS devices, which is not in itself much of an issue. Unfortunately, the backhaul for the link is a 4G modem on each end, neither of which has a publicly addressable IPv4 address. Still not an issue, I've managed to get it to talk using an EoIPv6 tunnel with each side pointed at it's opposite's cloud DNS name.

This works, and I'm able to talk over it. I am experiencing some odd behaviour on occasion, which I suspect is related to MTU issues. I've tried setting Mangle to clamp the MSS to 1400, but it hasn't seemed to make much difference. But, it works.

Unfortunately, I've run into a curve ball. I need to run BACNet/IP over the link which runs over UDP. Unfortunately, for BACNet, packet fragmentation happens at the application level and the devices I'm trying to talk between don't have the ability to handle it (Its designed for within a building, I'm trying to span a single BACNet network over the link for testing purposes). On top of that, the BACNet specification requires a minimum 1497 MTU, which is more than I can run over the EoIP tunnel.

Any ideas what I can do to get around this? I was wondering if I could somehow run VPLS as a layer over the EoIPv6 link, which I believe will handle the packet fragmentation - is that correct? I was thinking then I could put a Mangle rule on either end to clamp the MSS to something like 1300, then TCP connections shouldn't fragment in inopportune times but UDP will be able to pass a full ethernet packet? Does the MSS even work like that, on the same subnet? I don't know much about the MSS, to be honest.

Thanks!
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11982
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Site to site Layer 2 VPN with full ethernet MTU -- over IPv6

Tue Jul 13, 2021 2:44 pm

Disable "Clamp TCP MSS" and set MTU to 1500 on EoIP interface,
leave MikroTik do his works.
 
brendanp
just joined
Topic Author
Posts: 20
Joined: Sun Dec 04, 2011 6:07 am

Re: Site to site Layer 2 VPN with full ethernet MTU -- over IPv6

Thu Jul 15, 2021 5:31 am

Thanks for the reply rextended!

I've disabled the clamp MSS, the EoIP interface was already set to 1500, however I'm still seemingly unable to pass traffic larger than 1416, as tested with "ping -f -l <size> <remote host>". For sizes up to and including 1416 the ping replies are coming through correctly. From 1417 and higher, requests are being timed out. Additionally, TCP connections aren't entirely stable - to copy the config backup from the remote router to my machine, I had to temporarily set the bridge MTUs on both routers to 1400 and change my PC's MTU to make the file transfer successfully.

Rather than posting the whole config of both routers, I've condensed all of the config to just the parts applicable to the tunnel. The firewall filter rules are all at the top of the chain so all those three rules are being applied first. I can post the full exports if required but I didn't want to just dump it all here...

Both devices are running on Stable and up to date 6.48.3
/interface eoipv6
add ipsec-secret="BigLongSecretKey" local-address=:: loop-protect-disable-time=0s loop-protect-send-interval=0s mac-address=00:11:22:33:44:55 mtu=1500 name=eoipv6-tunnel1 remote-address=oppositerouter.sn.mynetname.net tunnel-id=0

/interface bridge port
add bridge=bridge comment=defconf interface=ether1
add bridge=bridge comment=defconf interface=wlan1
add bridge=bridge interface=eoipv6-tunnel1 multicast-router=disabled

/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface-list=WAN

/ipv6 firewall filter
add action=accept chain=input dst-port=500,4500,1701 in-interface=lte1 protocol=udp
add action=accept chain=input in-interface=lte1 protocol=ipsec-ah
add action=accept chain=input in-interface=lte1 protocol=ipsec-esp
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11982
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Site to site Layer 2 VPN with full ethernet MTU -- over IPv6

Thu Jul 15, 2021 10:05 am

Something between block fragmented packet?

EoIPv6 for reach 1500 can fragment packet, but reassemble it on correct order on other side without the destination see the differences.

If something inside are encapsulated:
Leave disabled "Clamp TCP MSS" and set MTU to 1600,
if still no trffic, probably something forbid/block fragmented packets
 
brendanp
just joined
Topic Author
Posts: 20
Joined: Sun Dec 04, 2011 6:07 am

Re: Site to site Layer 2 VPN with full ethernet MTU -- over IPv6

Thu Jul 15, 2021 11:17 am

I just tried increasing the MTU on both sides to 1600, no change unfortunately. I assume you meant on the eoipv6 interfaces...? Or should I make that change somewhere else (I'm not familiar with the way Mikrotik handles MTU except to refrain from touching it as much as possible).

There's nothing else running on the router that should interfere - This is a single layer 2 network spanned across the ethernet ports on both routers, that's all that's configured. I noticed that the eoip (ipv4 version) has an option for "Dont Fragment" while the ipv6 version doesn't, does the ipv6 always fragment packets to fit, or perhaps does it never fragment packets to fit....?

I'll obfuscate any private information in the config when I get a chance and attach the configs for both sides (unless there's another reply in the meantime that fixes it!)
 
andriys
Forum Guru
Forum Guru
Posts: 1526
Joined: Thu Nov 24, 2011 1:59 pm
Location: Kharkiv, Ukraine

Re: Site to site Layer 2 VPN with full ethernet MTU -- over IPv6

Thu Jul 15, 2021 1:56 pm

MTU of the EoIP interface itself should always match the MTU of the networks you are bridging, i.e. 1500 in most cases.

In-transit fragmentation is forbidden in IPv6 networks, packets may only be fragmented by sending parties. Functional PMTUD is vital in IPv6, so make sure you do not block ICMPv6.
 
mada3k
Long time Member
Long time Member
Posts: 687
Joined: Mon Jul 13, 2015 10:53 am
Location: Sweden

Re: Site to site Layer 2 VPN with full ethernet MTU -- over IPv6

Fri Jul 16, 2021 9:44 am

What is the LTE MTU set to ?
 
brendanp
just joined
Topic Author
Posts: 20
Joined: Sun Dec 04, 2011 6:07 am

Re: Site to site Layer 2 VPN with full ethernet MTU -- over IPv6

Mon Jul 19, 2021 2:18 am

The LTE connection MTU is 1500 and is able to pass that size traffic from site to site:
[admin@Main] > /ping Field-Host-IPV6 do-not-fragment size=1500
  SEQ HOST                                     SIZE TTL TIME  STATUS
    0 Field-Host-IPV6 	 1500  62 74ms  echo reply
    1 Field-Host-IPV6 	 1500  62 110ms echo reply
    2 Field-Host-IPV6 	 1500  62 72ms  echo reply
    3 Field-Host-IPV6 	 1500  62 70ms  echo reply
    sent=4 received=4 packet-loss=0% min-rtt=70ms avg-rtt=81ms max-rtt=110ms

[admin@Main] > /ping Field-Host-IPV6 do-not-fragment size=1501
  SEQ HOST                                     SIZE TTL TIME  STATUS
    0                                                         packet too large and cannot be fragmented
    0 Main-Host-IPV6 	 1280  64 0ms   packet too big
    1                                                         packet too large and cannot be fragmented
    1 Main-Host-IPV6 	 1280  64 0ms   packet too big
    2                                                         packet too large and cannot be fragmented
    2 Main-Host-IPV6  	1280  64 0ms   packet too big
    sent=3 received=0 packet-loss=100%
I thought the ICMPv6 PMTUD comment was the answer until I checked it today, but that is already set up. I did some more googling and I came across the below forum post which someone has the same issue, so I guess it's a known issue with EoIPv6 and full frame packets which is a bother, since I thought that was kind of the point of EoIP.

I guess I'll have to do the IPIPv6 with EoIP over the top workaround. I'm happy to post configs if anyone is interested, but it sounds like a bug. Is there any point sending a supout to Mikrotik, would they bother looking into such a small, obscure bug which apparently has been around for a while....?

viewtopic.php?t=119305
 
andriys
Forum Guru
Forum Guru
Posts: 1526
Joined: Thu Nov 24, 2011 1:59 pm
Location: Kharkiv, Ukraine

Re: Site to site Layer 2 VPN with full ethernet MTU -- over IPv6

Mon Jul 19, 2021 8:42 pm

Is there any point sending a supout to Mikrotik....?
Yes, there is. Please do.
 
User avatar
webor
Frequent Visitor
Frequent Visitor
Posts: 97
Joined: Sat Dec 20, 2008 2:33 am
Location: Croatia, Europe

Re: Site to site Layer 2 VPN with full ethernet MTU -- over IPv6

Mon Jul 19, 2021 10:54 pm

Hi. I had similar requirement and the solution that worked for me was to perform BCP bridging over pptp tunnel and it worked ok enabling fullethernet MTU. Look at this guide: https://wiki.mikrotik.com/wiki/Manual:B ... _bridging) In my case one side had public ip so pptp was not an issues but in your case I am not sure about establishing the pptp tunnel as you said no public ips, but maybe you can try to derive something.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11982
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Site to site Layer 2 VPN with full ethernet MTU -- over IPv6

Tue Jul 20, 2021 12:06 am

For my headache also you suggest BCP?
 
User avatar
webor
Frequent Visitor
Frequent Visitor
Posts: 97
Joined: Sat Dec 20, 2008 2:33 am
Location: Croatia, Europe

Re: Site to site Layer 2 VPN with full ethernet MTU -- over IPv6

Tue Jul 20, 2021 1:35 am

For my headache also you suggest BCP?
I do not understand, what do you mean? Apologies if the suggestion is not good. It was just a hint that helped me once upon a time, nothing special..

Who is online

Users browsing this forum: eworm and 82 guests