Wireguard Best Approach (two non-public sites)?

I always thought one needed at least one public accessible WANIP ( either directly or through port forwarding on an ISP upstream router etc.) to achieve good connectivity between two sites.

However, several threads showed that one can do funky things with two routers with private IPs, via wireguard to connect.

Then of course we have BTH which is ideally suited for the case of ones home or single site having no public IP address available. Again, several threads I have seen also demonstrated that one can actually connect two sites over BTH.

I guess I am asking given two routers, two sites, no public-Ip address available, what is the optimal way to connect them via wireguard ( yes assume vps/cloud is not an option ) ( yes assume zerotier is not available or desired aka no other types of vpn etc. )

For additional scope limitations, assume that each router has one subnet that must reach the other subnet and there are two be only two devices on the wireguard subnet /30.

Thoughts?

I asked Grok v4.5x and in 49 seconds came up with a solution --- would you like to see what Grok 4.5x produced ? ... FYI, SpaceX [a Elon Musk enterprise that recently went public] uses a lot of Mikrotik gear ... Grok is owned by Musk but I am not sure under which corp label.

probably IPv6

Hi Mozerd, sure/please.... you can send it to me in an email.

You need a 3rd party VPN solution that uses UDP hole punching or similar techniques (ZeroTier, Tailscale, Netbird etc.) - there's no way around it.

sent to your llamaworks email address

If you find that the Grok solution works for you please publish it here so that everyone can enjoy :grinning_face:

This is the right answer but @anav won't like that because he hates IPv6 :stuck_out_tongue:. But really, if you are behind NAT (including CGNAT) on both sides, and your ISP has support for IPv6, then use IPv6.

Apart from that, this is an excellent article from Tailscale about the challenges that UDP NAT hole punching must overcome: How NAT traversal works.

If @anav insists on using IPv4 only, then it's much easier if the NAT in between are all implementing Endpoint-Independent Mapping. Of course, an external helper with public reachable IP address is still needed.

Well I hadnt thought of IPV6, its always an option if progress is not made.
So it would appear you are saying, each router must be looking for an endpoint of the other router ( as if both are clients).

It's explained in the Tailscale article I linked above. With both MikroTik routers being behind NAT, and you configure the WireGuard interface on router 1 to use UDP port p1, on router 2 to use UDP port p2. Router 1 sees the IPv4 address on its WAN interface as a1.b1.c1.d1, router 2 see its address as a2.b2.c2.d2.

It's obvious that on router 1, configuring the remote WG peer as having endpoint-address=a2.b2.c2.d2 and endpoint-port=p2 will not work. Same with configuring the WG peer on router 2 with endpoint-address=a1.b1.c1.d1 and endpoint-port=p1. Because when router 1 sends out WG packet with the source a1.b1.c1.d1:p1, the NAT in front of it (for example the ISP's CGNAT) will transform the source into w1.x1.y1.z1:q1. Similarly, with router 2, the source of the packets is changed from a2.b2.c2.d2:p2 to w2.x2.y2.z2:q2.

The ISP NAT on the router 1 side will only translate incoming packets with destination w1.x1.y1.z1:q1 (as return packets) back to a1.b1.c1.d1:p1, and the ISP NAT on the router 2 side will only translate incoming packets with destination w2.x2.y2.z2:q2 back to a2.b2.c2.d2:p2.

To be able to establish a WG handshake between the two routers, At least, either router 1 musts know what w2.x2.y2.z2 and q2 are, or router 2 musts know what w1.x1.y1.z1 and q1 are.

Knowing what w1.x1.y1.z1 or w2.x2.y2.z2 are is easy, you can even do it manually with websites like https://ifconfig.me/ip. The challenge is to figure out the correct value for q1 and/or q2!

It's easier if one or both NAT sides use Endpoint-Independent Mapping. For example, let's say that the router 1 side ISP implement EIM, then within some time period after router 1 sends out an UDP packet with source a1.b1.c1.d1:p1 to the internet and the ISP NATs the source to w1.x1.y1.z1:q1, every other UDP packets that router 1 sends out with a1.b1.c1.d1:p1 as source, regardless of destination (doesn't have to be the same destination as the previous packet) will also have the source translated to the same w1.x1.y1.z1:q1. The value q1 is the same, independent from the destination (hence Endpoint-Independent Mapping).

In that case, it's trivial to figure out what q1 is if we have a helper server with public accessible IP address (for example a STUN server). Router 1 sends UDP packet with source a1.b1.c1.d1:p1 to that helper service, and the server can see the value for q1 (and of course w1.x1.y1.z1 too). Router 2 can then set the endpoint of the WG peer to w1.x1.y1.z1:q1 and make handshake attempts. Due to using EIM, the NAT on the router 1 ISP side will translate the incoming packets with destination w1.x1.y1.z1:q1 to a1.b1.c1.d1:p1 and the packets will arrive at the WG port on router 1.

Even if we don't have a 3rd party public reachable STUN server, a trick can be employed where the two router sides make hundreds of attempts with different port numbers toward the other router. As explained with the birthday paradox, a few hundreds of tries are enough to have a 50% matching chance, and with 1000 probes the chance raises to 98%.

The big problem is when neither of the side have EIM NAT, only Endpoint-Dependent Mapping. This is a Symmetric NAT situation. The value of the mapped port q1 for example, not only depends on the source a1.b1.c1.d1:p1, but also on the destination of the UDP packet. Different destinations produce different q1 for the same p1.

In that case, the trick with the helper STUN server no longer works. Because the port q1 that the STUN server sees might be different from the mapped port number that router 1's ISP choose when router 1 tries to connect to router 2 (two different destinations).

The random probe method also becomes much more difficult, because the number of possibilities has squared (65K * 65K), according to the article, each side need to send 170K probes for a 99.9% success rate. Which might not even work, the ISPs might see the flooding as hacking attempts and block everything. Or, most probably, the ISP NAT mapping table has limited entries and will recycle the ports, which means a match might never be found, even with millions of tries. Also. you'll have to figure out how to do that probing with RouterOS script, or write programs that do the probing and run them on both sides on hosts with DMZ enabled.

That's why usually in case of Symmetric NAT after some unsuccessful probing, a 3rd party relay service will be used instead (like the MikroTik relay with BTH).


Conclusion: It's realistically doable without the help from public helper server only if the NAT on at least one side uses Endpoint-Independent Mapping.


Other note: If you are an ISP and use MikroTik devices, then don't forget that the default SRCNAT in RouterOS is not Endpoint-Independent. If you are ISP and use the normal masquerade or src-nat action then your customers might have issues caused by "Symmetric NAT". The NAT chains have a special action for endpoint-independent-nat that works only for UDP: NAT | RouterOS Manual. And normally you'll have rules for both directions, for example:

/ip firewall nat
add action=endpoint-independent-nat chain=srcnat \
    out-interface-list=WAN protocol=udp randomise-ports=no
add action=endpoint-independent-nat chain=dstnat \
    in-interface-list=WAN protocol=udp randomise-ports=no

This is not important if the router is only your home router.

Nice try but zing went over my head. I will have to reread several times.
My question is in functionality, compared to using BTH at router A, and normal wireguard (although seen as BTH client) on Router B--->, is endpoint independent nat, more or less stable, has more or less latency, has more or less throughput, is more or less secure???

PS Not that I didnt believe you but I had to verify if EIM was an even an option on firewall/nat as I had never seen it before, lo and behold there it was. I cannot keep up!!

Does this new functionality fit into the wonderful flow diagrams produced lately??

So in your construct, you would not use BTH but EIM with regular wireguard???

The EIM is not a setting for you to change on your devices, it's something that the provider in front of your devices that imposes NAT on you (the ISP that deploys CGNAT, or the hotel's / coffee shop's / residential building's router, etc...) support or not.

The NAT they use implement EIM, then it's easier for you to "punch holes" through it (figuring out the mapped UDP port), which in turn allow your two sides to make a direct WireGuard connection between them, without needing a relay. The performance and latency will then be the same as when one of the side has a public accessible IP address, the WG connection is a direct connection.

If instead both your sites are behind a Endpoint-Dependent Mapping NAT (again this is not a setting you can change on your router, but something that the people who provide the internet connection to you decide), then you have a "Symmetric NAT" setup and it's very difficult to figure out the port mapping. As a result, you probably won't be able to establish a direct WG connection between the two sites (UDP packets cannot be sent directly between the two sites), a relay server will be needed, and you'll have increase latency and lower throughput (depending on the current load on the relay server).

Ahh okay, so EIM is a functionality only to be used if both ISPs do not have symmetric NAT on their service.
So what is the easiest way to determine the type of NAT an ISP has? Above you stated both sides need it but in the previous post, you stated at least one side needs it?

No, EIM is something you hope that your ISP is using if they put you behind NAT. Because in that case, even if you are behind NAT you still have the chance to have direct connections without relay when using services like BTH, Tailscale, Zerotier, etc...

If you want the most correct result, you'll need to plug a Playstation or XBOX console directly to the ISP router and use their NAT testing feature. Or if you must have a MikroTik router in front, then you'll need to temporary configure that the router to DMZ the console during the test. Otherwise, the test will not be correct because you have the firewall of your router that does NAT too in front of the console.

There are several websites that also let you test the NAT type using your browser, but they will be less accurate, because your PC is also behind your router (that does NAT) and the operating system on the PC also has a firewall that does filtering and will cause the result to be "Port-Restricted NAT" or "Symmetric NAT" in most cases. If you want to test the true NAT type of you CGNAT ISP, you would need to plug the PC directly to the modem and temporarily disable its firewall.

Or you can setup Zerotier on the MikroTik router an see if it's able to do UDP hole punching. If yes, then you don't suffer from symmetric NAT.

@CGGXANNX 's explanations are spot on.

For a bit different perspective, any time you want to ensure that you can correctly accept inbound connections, which you have to be able to do in your proposed scenario on at least one end, you usually do one of port forwarding (probably the cleanest) or you use upnp or nat-pmp to achieve the same.

The problem is that ISPs are reluctant to permit any of this when they install cgnat. For upnp and nat-pmp this is somewhat rational for various security reasons, but it's s bit more baffling to me why they don't give everyone - let's say - 10 public forwarded (arbitrarily assigned) ports. This would solve most people's problems. But this is an aside.

Endpoint-independent nat is a bit contrived solution to this problem. It allows approaches like STUN and TURN to work. And there are lots of free/public services providing this, like Google's. Read up on stun if you want to know more, sadly the Wikipedia article is... lacking.

So sadly the answer to your question is that if you want to do this in a fully compatible way, the only way is to have an external relay. It's sort of important to mention it here that in most cases involving wireguard and relays (this includes BTH, Tailscale and Netbird) the relay only helps with networking and forwards an encrypted stream that they don't have the keys for. So such a relay doesn't impact the security properties of wireguard.

I really wish that Mikrotik would allow either private BTH relays, or provide a built-in (non-container) client implementation for one of tailscale or netbird. One can always hope.

@CGGXANNX is also right in pointing out that multiplayer games are one of the big users of these nat traversal technologies, mainly because of reduced latency and the cost of operating servers. They will tell you what sort of nat is in front of them, but so will any number of purpose-built tools.

Generally home routers and ones in hotels/corporate will not provide endpoint-independent nat, however most cgnat implementations will. This still doesn't really help with what you ask, because this is still predicated on what your ISP or other provider does, so it's far from being a universal solution...

P.S. And yes, if you find yourself in the position of being a cgnat provider for others, you should absolutely take the effort of using ein-nat.

The Easiest Test: WAN IP vs. Public IP Comparison

  1. Log into your router’s admin dashboard and locate your WAN IP (or Internet IP).

  2. Open a web browser on a device connected to your Wi-Fi and visit an IP checking site like WhatIsMyIP or IPLocation.

  3. Compare: If the WAN IP in your router matches the Public IP on the website, you have a direct Public IP. If they do not match, your ISP is using CGNAT.

Router IP Address Check

If your router’s WAN IP starts with any of the following, your ISP is definitely using CGNAT or multi-level NATs:

  1. 100.64.x.x through 100.127.x.x (Dedicated CGNAT range)
  2. 10.x.x.x, 172.16.x.x, or 192.168.x.x (Private local ranges)

Curtesy of Google

BTW, Anav have you considered the Grok approch?

Not yet Sir, real life getting in the way here LOL. Chainsaw work to cut down trees, mow lawn, family obligations, and thats only the start ( lots to do before the cold wx and snow come back LOL ). Being retired means doing all the things you were doing before (routine) but now have no excuse do all the things you have spent money preparing for but not even started ( can I say patch panel revamp from a wooden box into a proper standing closet rack!!

Yep, time waits for no one :grinning_face_with_smiling_eyes:

BTW, I upgraded my CCR1009 to the ccr2004-16g-2s+pc and so far am very pleased with its performance using RoS v7.23.1 ... my ipv6 'seems' much faster on this device

Funny the same upgrade I did awhile back. Good price at ISP Supplies.

I purchased from getic in Latvia …

?? Visiting or Huge Discount LOL