Community discussions

MikroTik App
 
vitovanjak
just joined
Topic Author
Posts: 7
Joined: Fri Feb 02, 2018 10:18 am

[WireGuard] It works, but why?

Mon Nov 21, 2022 12:24 am

Hi,

I have a WG server on Ubuntu with Hetzner VPS. I forwarded(routed, not NAT) the public floating IP(123.123.123.123) to the Mikrotik router(WG client) with the following configuration:

WG Server(Ubuntu):
[Interface] Address = 10.0.0.1/24
ListenPort = 51820
PrivateKey = <server private key>

[Peer]
PublicKey = <client public key>
AllowedIPs = 10.0.0.2/32, 123.123.123.123/32
WG Client(MikroTik):
/interface wireguard
add listen-port=13230 mtu=1420 name=ToHetznerVPS

/interface wireguard peers
add allowed-address=0.0.0.0/0,::/0 endpoint-address=<server primary ip> endpoint-port=51820 interface=ToHetznerVPS public-key="<server public key>"

/ip address
add address=123.123.123.123 interface=ToHetznerVPS network=123.123.123.123
add address=10.0.0.2 interface=ToHetznerVPS network=10.0.0.2
This all work and all request(HTTP) and pings come directly to MikroTik and then I NAT them from there.

The problem is that my Avg. traffic on the WG interface is around 1 Mbps and on my WAN interface is 20 Mbps which is more realistic for the website I am hosting.
Image
Can it be that requests are coming through WG but the response is directly over my WAN and it is not going through the tunnel back? (I think that is not possible but I am confused at the moment, so I ask)
I expect that my WAN traffic is bigger or equal to Wireguard traffic.

Also when I torch(monitor) traffic on the WAN interface I can see, a lot of requests with a destination address of the VPS floating IP(123.123.123.123)

Can someone explain what is going on here?
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: [WireGuard] It works, but why?

Mon Nov 21, 2022 5:28 pm

Full config of mikrotik required
/export file=anynameyouwish ( minus router serial number and any public WANIP information )
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: [WireGuard] It works, but why?

Mon Nov 21, 2022 5:32 pm

Your requirements are not clear.
I get the wireguard part in that you wish to be able to connect the MT to the Ubuntu but why??

A. Do users on MT need access to subnets on ubuntu?
B. Do users need to access internet via ubuntu
C. Are there users or devices on Ubuntu that will originate traffic with destination on Mikrotik?
D. Will you as admin want to wireguard into UBUNTU to reach MT servers/subents or perhaps to config mikrotik from remote location (hotel or cafe via laptop or iphone)?
E. Do you have other remote users connecting via wireguard to ubuntu that require access to MT subnets.

I have no clue what you are attempting to do with 123.123.123
And also dont know if its really feasible or a good idea to give the wireguard IP address and 123.123.123 address to the same wireguard interface name ????


Okay, what is your intention with 123.123.123 Is that another WAN connection at the ubuntu?
What did you have in mind for this second WAN connection?

IF its not a WAN connection then what is it???
 
Sob
Forum Guru
Forum Guru
Posts: 9119
Joined: Mon Apr 20, 2009 9:11 pm

Re: [WireGuard] It works, but why?  [SOLVED]

Mon Nov 21, 2022 8:13 pm

Can it be that requests are coming through WG but the response is directly over my WAN and it is not going through the tunnel back? (I think that is not possible but I am confused at the moment, so I ask)
It's actually very possible and default behaviour. Unless you made sure that everything from 123.123.123.123 is routed via WG, it will use default route. The part that confuses me is that you'd notice this not just by looking at counters, but because communication between 123.123.123.123 and outside world wouldn't work at all (unless your ISP does very bad job filtering spoofed addresses). So there must be something more to it.

@anav: It's the "how to give public address to WG client, because it didn't get any from own ISP" scenario.
 
vitovanjak
just joined
Topic Author
Posts: 7
Joined: Fri Feb 02, 2018 10:18 am

Re: [WireGuard] It works, but why?

Mon Nov 21, 2022 8:20 pm

Hi,

Config export is in the attachment I hope I didn't leak any private info ;)

A. Nope, they don't
B. No, they don't
C. Not sure I understand the question
D. Not sure I understand the question
E. No, only one

I will try to explain.
So just to be clear everything is working as I am expecting, I am just surprised that the WG interface doesn't show more traffic.

I have a MikroTik router with a dynamic IP address from my ISP that has 4 days lease. I wanted a public static IP address so I can host different things with minimal downtime.
So my idea was to rent VPS with a floating IP address and I did that.
On my VPS I have a primary IP address and I rented one more floating IP address and didn't mount it to the interface so I can use it as a WG peer address.

DDNS was not an option for me because on IP change there is usually around 1 min provisioning time.
With WG on IP change, I have around 3 seconds of downtime because it doesn't require a handshake so it is very fast!

So now with this configuration when I ping or HTTP my floating IP address all connections come directly to my MK router and then from there, I can do whatever I want (NAT)

So Ubuntu VPS is just kinda relay for public static IP address and that's it.

Note: I didn't show it in config but I have 5 floating IP addresses and all of them I routed through WG so I can use it for different web servers I have and it works great.

Hope my explanation helps.
You do not have the required permissions to view the files attached to this post.
 
vitovanjak
just joined
Topic Author
Posts: 7
Joined: Fri Feb 02, 2018 10:18 am

Re: [WireGuard] It works, but why?

Mon Nov 21, 2022 8:23 pm

@anav: It's the "how to give public address to WG client, because it didn't get any from own ISP" scenario.
Exactly, my ISP won't provide me with static IP so this is a workaround
 
Sob
Forum Guru
Forum Guru
Posts: 9119
Joined: Mon Apr 20, 2009 9:11 pm

Re: [WireGuard] It works, but why?

Mon Nov 21, 2022 8:57 pm

It shouldn't work at all (in today's internet). You need to treat it as regular dual WAN, unless you'd be using WG as default and main WAN only for connecting to WG server. So another routing table, in it default route with WG interface as gateway, mark incoming connections from WG, mark routing for responses to go out via WG.
 
vitovanjak
just joined
Topic Author
Posts: 7
Joined: Fri Feb 02, 2018 10:18 am

Re: [WireGuard] It works, but why?

Mon Nov 21, 2022 9:06 pm

It shouldn't work at all (in today's internet). You need to treat it as regular dual WAN, unless you'd be using WG as default and main WAN only for connecting to WG server. So another routing table, in it default route with WG interface as gateway, mark incoming connections from WG, mark routing for responses to go out via WG.
Gotcha, so I had a good hunch that something is weird. Can you maybe help me with settings this up?

I think I need to do this but I am not exactly sure how to mark connection and mark routing.
/routing table
add disabled=no fib name=wireguard-table
/ip route
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=ToHetznerVPS \
    pref-src="" routing-table=wireguard-table scope=30 suppress-hw-offload=no \
    target-scope=10
 
Sob
Forum Guru
Forum Guru
Posts: 9119
Joined: Mon Apr 20, 2009 9:11 pm

Re: [WireGuard] It works, but why?

Mon Nov 21, 2022 9:25 pm

Correct. Follow with:
/ip firewall mangle
add chain=prerouting in-interface=ToHetznerVPS connection-state=new action=mark-connection new-connection-mark=WG_conn
add chain=prerouting in-interface=LAN-CLOUD connection-mark=WG_conn action=mark-routing new-routing-mark=wireguard-table
And add connection-mark=no-mark to existing fasttrack rule.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: [WireGuard] It works, but why?

Mon Nov 21, 2022 9:30 pm

ONce done maybe Sob will put this in simple terms so I can understand LOL............
Would be a great addition to the article.
For example the public IP is on the VPS and not at the Mirkotik..........
So the VPS gets the public IP and how is that passed to the mikrotik - I didnt think you could have two public IPs from one connection?
One to one NAT ????? VPS public IP to one MT private IP
Normal NAT????? VPS public IP natted to anynumber of MT private IP/subnets........

So we have one IP address U-A, which is used to establish the wireguard tunnel between the UBUNTU VPS and the Mikrotik.
THe OP has X number of other floating IP addresss U2-U5 that he wants to relay through the tunnel to the mikrortik for what purpose??

AS WAN2, WAN3, WAN4 and WAN5 so to speak..........

/interface list members
add ether1-local-WAN list=WAN
add ?????? list=WAN
add ?????? list=WAN
add ????? list=WAN
add ???? list=WAN

As you can see, thoroughly confused LOL
 
vitovanjak
just joined
Topic Author
Posts: 7
Joined: Fri Feb 02, 2018 10:18 am

Re: [WireGuard] It works, but why?

Tue Nov 22, 2022 12:07 am

Thank you Sob that did the trick.
Image

Even do on Hetzner VPS I have a bandwidth limit of 20TB so now I have to keep eye on that, I will use it without a custom routing table while it works... If one day stops working I will fall back to the custom routing table method.
Image
Hetzner Graph
 
vitovanjak
just joined
Topic Author
Posts: 7
Joined: Fri Feb 02, 2018 10:18 am

Re: [WireGuard] It works, but why?

Tue Nov 22, 2022 12:30 am

ONce done maybe Sob will put this in simple terms so I can understand LOL............
Would be a great addition to the article.
For example the public IP is on the VPS and not at the Mirkotik..........
So the VPS gets the public IP and how is that passed to the mikrotik - I didnt think you could have two public IPs from one connection?
One to one NAT ????? VPS public IP to one MT private IP
Normal NAT????? VPS public IP natted to anynumber of MT private IP/subnets........

So we have one IP address U-A, which is used to establish the wireguard tunnel between the UBUNTU VPS and the Mikrotik.
THe OP has X number of other floating IP addresss U2-U5 that he wants to relay through the tunnel to the mikrortik for what purpose??

AS WAN2, WAN3, WAN4 and WAN5 so to speak..........

/interface list members
add ether1-local-WAN list=WAN
add ?????? list=WAN
add ?????? list=WAN
add ????? list=WAN
add ???? list=WAN

As you can see, thoroughly confused LOL
I can see you didn't give a look at the configuration export that I attached. I think the configuration is very simple so you can read it very easily. The only change that I did from the normal configuration that you can find on the internet is that I gave peer public IP instead of private IP and that's it.

VPS has one main IP that terminates at Ubuntu and I can SSH to Ubuntu over it. And then I have additional public IPs that I rented from Hetzner called Floating IPs.

Just give the configuration a look I think you will understand it quickly ;)
 
Sob
Forum Guru
Forum Guru
Posts: 9119
Joined: Mon Apr 20, 2009 9:11 pm

Re: [WireGuard] It works, but why?

Tue Nov 22, 2022 1:32 am

Just give the configuration a look I think you will understand it quickly ;)
Well, about that...

(sorry, I couldn't resist :D; I'll write a bit more info later)
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: [WireGuard] It works, but why?

Tue Nov 22, 2022 10:56 pm

Waiting.......... tap tap tap.................
 
User avatar
k6ccc
Forum Guru
Forum Guru
Posts: 1490
Joined: Fri May 13, 2016 12:01 am
Location: Glendora, CA, USA (near Los Angeles)
Contact:

Re: [WireGuard] It works, but why?

Wed Nov 23, 2022 2:12 am

I didnt think you could have two public IPs from one connection?
Yes, you can - I did it for several years. My DSL provider gave me up to eight static public addresses. My then RG750r2 had one WAN interface that was using four or five of the public IPs. Each public IP was routed to a different LAN. Essentially it was set up as if each LAN / Public IP pair was a separate simple consumer type router (which is what it replaced). Had to re-arrange it when my DSL provider stopped providing multiple static IPs to residential service.
 
Sob
Forum Guru
Forum Guru
Posts: 9119
Joined: Mon Apr 20, 2009 9:11 pm

Re: [WireGuard] It works, but why?

Thu Nov 24, 2022 4:16 am

Patience, young anav! Sob gotta eat. And work, annoying prerequisite. ;)

About this thing with addresses, the basic idea is that public addresses are good, because they allow to host servers and such, which is fun. Ideally, everyone should have enough public addresses, but until IPv6 takes over, it's currently problematic. So let's assume that you want nice static public IPv4 address, but your ISP gives you only some not good enough (dynamic, NAT 1:1), or none at all. But you may be able to find better one elsewhere (rent a VPS, ...). Then you can choose the simple and lazy way, make a VPN with private addresses and only do dstnat from that public address to internal one behind VPN. But where's the fun in that? If you got nice shiny public address, you want it on your router (or directly on internal server).

WG part of this is nothing special. Downright boring, really. On VPS/server, if it's RouterOS:
/interface wireguard
add name=WG listen-port=<server port> ...
/interface wireguard peers
add interface=WG allowed-address=<public address>/32 ...
/ip route
add dst-address=<public address> gateway=WG
On client:
/interface wireguard
add name=WG-WAN ...
/interface wireguard peers
add interface=WG-WAN allowed-address=0.0.0.0/0 endpoint-address=<server address> endpoint-port=<server port> ...
/ip address
add interface=WG-WAN address=<public address>/32
plus the usual dual-WAN config, because WG-WAN is same as another regular WAN interface. I told you it's boring.

The more interesting part is what you do on server side. Did you notice <public address> and <server address> in previous example? Yes, it's two addresses, one used by server and the other for you. There are different ways how to get it. It can be routed to server by ISP. So server has <server address> and ISP does:
/ip route
add dst-address=<public address> gateway=<server address>
which means that any traffic to <public address> is routed to <server address> and server can do anything it wants with it (route it further as in example). It's the "floating IP" from this thread.

Another way is that both addresses are from same subnet and what ISP meant is to assign them both to connected device. If it's RouterOS, it would be:
/ip address
add interface=WAN address=<server address>/<mask>
add interface=WAN address=<public address>/<mask>
But no, you want the second address on the other router! No problem, don't add <public address> as address, instead use proxy ARP to lie to ISP that it's there:
/ip arp
add interface=WAN address=<public address> published=yes
Done. Now everything to <public address> will be sent to your VPS/server/router and you can route it further via VPN to second one.

But it's not good. You need one address, but you had to get two. And you know that they'll charge extra for it. If only you could do it with one, you don't want address on server (VPS), you want it on your router (client). But you must have public address on VPS, otherwise you wouldn't be able to connect to it. And you can't have same address in two places, right? Wrong, of course you can (for recent v7; would be different for older versions, plus v6 couldn't use WG):
/ip address
add interface=WAN address=<shared public address>/<mask>
/interface wireguard
add name=WG listen-port=<server port> ...
/interface wireguard peers
add interface=WG allowed-address=<shared public address>/32 ...
/routing table
add name=to-client fib
/ip route
add dst-address=<shared public address> gateway=WG routing-table=to-client
/ip firewall mangle
add chain=prerouting dst-address=<shared public address> connection-state=new protocol=udp dst-port=<server port> action=accept
add chain=prerouting dst-address=<shared public address> connection-state=new action=mark-connection new-connection-mark=forward_conn passthrough=yes
add chain=prerouting connection-mark=forward_conn in-interface=WAN action=mark-routing new-routing-mark=to-client
So any connection to <shared public address>:<server port> (WG server) will be just accepted and will go to this device. But any other connection to <shared public address> will be marked and routed to second router (WG client). Hooray, success! Well, almost, I forgot that second router connecting to remote <shared public address> when it has same "own" local <shared public address> won't work well, or more precisely not at all. If it has RouterOS, it's probably doable, but I'll have to test it... later, because now I'm almost asleep.

--

Edit: About this last part with single shared address, it seems that I went too far. It's not impossible. But with UDP-based roaming-happy WG it's real PITA. Basic idea sort of works. Connection to WG server (<shared public address>:<server port>) is forced to use separate routing table with gateway being local ISP:
/interface wireguard
add name=wg-wan ...
/routing table
add name=to-server fib
add name=wg fib
/interface wireguard peers
add interface=wg-wan allowed-address=0.0.0.0/0 endpoint-address=<shared public address> endpoint-port=<server port> persistent-keepalive=10s public-key="xxx"
/ip address
add address=<shared public address> interface=wg-wan
/ip firewall mangle
add chain=output dst-address=<shared public address> protocol=udp dst-port=<server port> action=mark-routing new-routing-mark=to-server passthrough=no
add chain=prerouting in-interface=wg-wan connection-state=new action=mark-connection new-connection-mark=wg_conn passthrough=no
add chain=output connection-mark=wg_conn action=mark-routing new-routing-mark=wg passthrough=no
add chain=prerouting in-interface=!wg-wan connection-mark=wg_conn action=mark-routing new-routing-mark=wg passthrough=no
/ip route
add dst-address=<shared public address>/32 gateway=<local gateway> routing-table=to-server
add dst-address=0.0.0.0/0 gateway=wg-wan routing-table=wg
It works for initial packet that chooses local address as source. But then it realizes that best source for connecting to <shared public address> is actually <shared public address> and switches to that. So it needs some srcnat to fix it:
/ip firewall nat
add chain=srcnat dst-address=<shared public address> protocol=udp dst-port=<server port> action=masquerade
But then it still goes wrong, addresses and ports get somehow mixed up. And honestly, I lost my patience. It would work with something based on TCP, or if WG could be forced to bind to specified local address.
Last edited by Sob on Fri Nov 25, 2022 3:12 am, edited 2 times in total.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: [WireGuard] It works, but why?

Thu Nov 24, 2022 12:56 pm

Patience, young anav! Sob gotta eat. And work, annoying prerequisite. ;)
So you typed this with a chrome notebook whilst sitting on the great white telephone?
Much thanks will look at it later as I have similar time constraints. :-)

Who is online

Users browsing this forum: Bing [Bot], GoogleOther [Bot], holvoetn, kazza, Soleous75 and 73 guests