Community discussions

MikroTik App
 
DL7JP
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 85
Joined: Sat Oct 19, 2013 4:14 pm

Wireguard bug: connections via WG tunnels suddenly failing

Fri May 14, 2021 6:57 pm

I am experimenting since 3 monhts or so with the wireguard implementation running on a RB450G. It it works, it works like a charm, but I regularly see clients suddenly failing to route via the tunnel, without having touched the condiguration on either side. The incomming connection is shown in the server, the tx counter on the client increases, but rx stays at 92 byte, after a few seconds 120 byte, etc. Whatever I tried did not fix this, only deploying a new client key pair helped.

Meanwhile I found a simpler way: If I change e.g. one character of the client's public key on the server, safe the key, and then change it back to the original, correct value and safe it, all works fine again. It seems like the internal representation of the client's public key on the server becomes somehow corrupted after a non-deterministic time. I cannot reproduce this phenomenon, and it also does not happen regularly.

Maybe this observation helps whoever is in charge of the wireguard implementation.
 
User avatar
kiler129
Member
Member
Posts: 352
Joined: Tue Mar 31, 2015 4:32 pm
Location: IL, USA
Contact:

Re: Wireguard bug: connections via WG tunnels suddenly failing

Sun May 16, 2021 6:43 am

I'm currently debugging something similar. Couple questions to you:

1. Is disabling the WG interface and re-enabling it again fixes the problem?
2. Can RB ping the client in this broken state?
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: Wireguard bug: connections via WG tunnels suddenly failing

Tue May 18, 2021 2:42 pm

I just started using WG tunnel between an RB450Gx4 acting as a server behind a CCR1009 router and the other end is an RB4011 behind a consumer router.
Not enough experience to know if this happens. what specific log entry can be made to pinpoint if this happens?
Otherwise way to much noise on logs??
 
DL7JP
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 85
Joined: Sat Oct 19, 2013 4:14 pm

Re: Wireguard bug: connections via WG tunnels suddenly failing

Wed May 19, 2021 10:35 pm

I'm currently debugging something similar. Couple questions to you:

1. Is disabling the WG interface and re-enabling it again fixes the problem?
2. Can RB ping the client in this broken state?
Sorry, I am not here too often ... as to 1, I did not try this, will do so when it happens next time; however, since a reboot did not solve the problem, I guess the answer here is "no". The answer to 2 is "no".
Last edited by DL7JP on Thu May 20, 2021 4:08 pm, edited 1 time in total.
 
DL7JP
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 85
Joined: Sat Oct 19, 2013 4:14 pm

Re: Wireguard bug: connections via WG tunnels suddenly failing

Wed May 19, 2021 10:38 pm

what specific log entry can be made to pinpoint if this happens?
Otherwise way to much noise on logs??
There seems to be no specific logging topic for wireguard - in fact I haven't seen any usefol log entry in this case. Debugging wireguard connections is really tough...
Last edited by DL7JP on Thu May 20, 2021 4:07 pm, edited 1 time in total.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: Wireguard bug: connections via WG tunnels suddenly failing

Thu May 20, 2021 12:28 am

I also just added my iphone as a wireguard client to my server and the MT app works great over that.
 
User avatar
kiler129
Member
Member
Posts: 352
Joined: Tue Mar 31, 2015 4:32 pm
Location: IL, USA
Contact:

Re: Wireguard bug: connections via WG tunnels suddenly failing

Fri May 21, 2021 12:20 am

Sorry, I am not here too often ... as to 1, I did not try this, will do so when it happens next time; however, since a reboot did not solve the problem, I guess the answer here is "no". The answer to 2 is "no".
I usually perform the following ritual when wg acting as a "client":
1. Disable/enable WG interface
2. Ping the WG endpoint/server
3. Ping the internal IP which should go over the tunnel

...and the tunnel magically comes back.
There seems to be no specific logging topic for wireguard - in fact I haven't seen any usefol log entry in this case. Debugging wireguard connections is really tough...
This is a problem with WG overall, not just on MT. As they essentially just shoot the packets over UDP hoping for the best even the WG itself has little to no knowledge about the tunnel... there's no "connection" or a "session". It's a blessing and a curse.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: Wireguard bug: connections via WG tunnels suddenly failing

Fri May 21, 2021 1:00 am

Sorry, I am not here too often ... as to 1, I did not try this, will do so when it happens next time; however, since a reboot did not solve the problem, I guess the answer here is "no". The answer to 2 is "no".
I usually perform the following ritual when wg acting as a "client":
1. Disable/enable WG interface
2. Ping the WG endpoint/server
3. Ping the internal IP which should go over the tunnel

...and the tunnel magically comes back.
There seems to be no specific logging topic for wireguard - in fact I haven't seen any usefol log entry in this case. Debugging wireguard connections is really tough...
This is a problem with WG overall, not just on MT. As they essentially just shoot the packets over UDP hoping for the best even the WG itself has little to no knowledge about the tunnel... there's no "connection" or a "session". It's a blessing and a curse.
Well the only reason it doesnt work for me is when I have an incorrect configuration. My limited knowledge in networking and vpns doesnt help LOL.
The best tools are sniffing traffic on ports along the various interfaces as well as ones log (assuming key firewall rules were set to be logged).
 
DL7JP
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 85
Joined: Sat Oct 19, 2013 4:14 pm

Re: Wireguard bug: connections via WG tunnels suddenly failing

Mon May 24, 2021 2:27 pm

I usually perform the following ritual when wg acting as a "client":
1. Disable/enable WG interface
2. Ping the WG endpoint/server
3. Ping the internal IP which should go over the tunnel

...and the tunnel magically comes back.
The problem I described was with the Mikrotik router being the WG server, clients are diverse (Andorid, IPhone, Win10); the problem is not bound to a specific client.
 
craftyjon
just joined
Posts: 1
Joined: Tue Sep 05, 2023 5:10 am

Re: Wireguard bug: connections via WG tunnels suddenly failing

Tue Sep 05, 2023 5:23 am

Meanwhile I found a simpler way: If I change e.g. one character of the client's public key on the server, safe the key, and then change it back to the original, correct value and safe it, all works fine again. It seems like the internal representation of the client's public key on the server becomes somehow corrupted after a non-deterministic time. I cannot reproduce this phenomenon, and it also does not happen regularly.
Just wanted to say thanks for documenting this, I was pulling out my hair trying to understand why my wireguard config wasn't working after moving it from one router to another. The symptoms were exactly as you describe, and I found that applying the same fix (changing each client's public key and then changing back to the original) fixed the issue for me also. I'll see if this issue comes back after a while.

Who is online

Users browsing this forum: No registered users and 24 guests