Community discussions

MikroTik App
 
tiernano
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 85
Joined: Fri Jul 20, 2012 1:51 pm

Multiple Wireguard tunnels using different WAN links

Thu Jun 16, 2022 5:16 pm

Morning all.

I have a CHR with 2 WAN links and 1 LAN link. Its primary use is tunneling and BGP. Without adding the second WAN link, the setup was as follows:
  • WAN link route was added to main.
  • 3 Wireguard tunnels to upstream servers with a /30 IP range for each tunnel (3 in total)
  • BGP connection to the upstream server. All learned routes were put into a table named ASN.
  • all traffic from router (wireguard, dns, etc) went direct over the WAN link.
  • routing rule for traffic over the LAN link: lookup using the ASN table (all devices on LAN get a public IP from my IPv4 or IPv6 space).
All works perfectly.

But, when adding a second WAN link, things dont work so well:
  • added second default route to main now pointing at second gateway IP
  • created 2 new tables, ISP1 and ISP2
  • 2 more defaults, now with ISP1 and ISP2 and their respective gateways
  • Have tried using a mangle rule, output chain, mark routing to given table, for a given dest IP
  • updated wireguard servers to send traffic to new IP
but the connection wont come up... looks like its sending traffic over the new link but with the old IP. I do have a pref-source with the static ip for each ISP, but even without it, it doesnt connect...

So, what am i missing? Am i doing this correct, or is there a better way? Thanks.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Multiple Wireguard tunnels using different WAN links  [SOLVED]

Thu Jun 16, 2022 10:07 pm

So, what am i missing?
What you are missing is that when the router sends a packet, it first finds a route for it, then assigns the IP address of the gateway interface of that route to the packet, and only as the last step it passes the packet through mangle/output. If a routing-mark gets assigned in mangle/output, the routing is repeated, but that new round of routing does not change the source address of the packet assigned in the previous round. So to change it, you need to use a src-nat or masquerade rule, even though we deal with router's own traffic here.

Yet another possibility might be to use a /routing/rule row rather than a mangle rule, as it is enough to match on destination addresses. That should cause the proper table to be used already during the first (and thus only) round of routing.
 
tiernano
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 85
Joined: Fri Jul 20, 2012 1:51 pm

Re: Multiple Wireguard tunnels using different WAN links

Thu Jun 16, 2022 10:25 pm

PERFECT! This is exactly what i needed! just updated my route rules, and now i can set each wireguard instance to a given connection! Happy days! Thank you!
 
iNaik
just joined
Posts: 8
Joined: Mon Oct 24, 2022 2:33 pm

Re: Multiple Wireguard tunnels using different WAN links

Mon Oct 24, 2022 5:13 pm

Hi,
I'm very interested to have 2 wireguards with 2 ISPs on the same Mikrotik.
I've tried to search online but the configurations I found, seems to be very complex.

My network is basic, with 2 ISPs on the WAN side, and some networks on LAN side. All the LAN go out to internet with ISP1, and the second ISP (ISP2) is only for failover.
I've a wireguard network with ISP1 working very good.

Then I create a second wireguard, but when I put the IP/DNS to the client side, it will not connect.
If I use the ISP2 as "main" route, the second wireguard conect perfect.

Can you make a little example of where and what I need to configure to connect the second wireguard with ISP2 when is not "main" (only for use for failover)

PD: My failover work with recursive addresses. I've RouterOS 7.6 on RB4011 (ARM)
Thanks Niks
Thanks Nico
 
tiernano
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 85
Joined: Fri Jul 20, 2012 1:51 pm

Re: Multiple Wireguard tunnels using different WAN links

Mon Oct 24, 2022 6:46 pm

So, for me, my IPs from my ISP are static. So, i manually setup the default routes. I only one default route, pointing at my main ISP. I then have 3 route tables setup: main (set by default) ISP1 and ISP2. then in routes, i have a second copy of my primary ISP default route, set with the ISP1 table. Then my secondary connection has ISP2 as its route table. the magic mainly happens in the route/rules section. In here, i set the IP of my first wireguard box to lookup only in the main route table, then set the second box to use the ISP2 table...

Not currently at the router, but hopefully this gives you some ideas...
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Multiple Wireguard tunnels using different WAN links

Mon Oct 24, 2022 7:22 pm

@iNaik, something in your post makes me think that your application scenario differs from @tiernano's one - in particular, that you want to use Wireguard to let external devices in the internet (like your phone and your laptop) get access to your LAN subnets, and that you want to be able to allow these external devices to connect to either of the two WAN addresses. Is this a correct understanding?
 
iNaik
just joined
Posts: 8
Joined: Mon Oct 24, 2022 2:33 pm

Re: Multiple Wireguard tunnels using different WAN links

Mon Oct 24, 2022 7:52 pm

The ip of my WireGuard is: 10.1.20.1/23 (image IP-WG)
I've created a separate table with name "rFIBRA-VODA" with the gateway of the ISP (in my case is pppoe conection) (image route-ISP2)
And create a Routing/rule for move all the traffic of the WG with the ISP2. (image Routing/rules).
But didn't work.

Any idea?
 
iNaik
just joined
Posts: 8
Joined: Mon Oct 24, 2022 2:33 pm

Re: Multiple Wireguard tunnels using different WAN links

Mon Oct 24, 2022 7:58 pm

@iNaik, something in your post makes me think that your application scenario differs from @tiernano's one - in particular, that you want to use Wireguard to let external devices in the internet (like your phone and your laptop) get access to your LAN subnets, and that you want to be able to allow these external devices to connect to either of the two WAN addresses. Is this a correct understanding?
Yesss,
My problem is that the ISP1 some times have problems with internal network, and some networks are not accesible depending where i connect (example, if i go to other client of the same ISP1, the WG didn`t work, and clients with other ISPs or 4g/5g netwotks, works.

My idea is have 2 WG, and if the frist (most used with ISP1) fails, use the second WG to access to my LAN devices.
If is possible, have one WG and create 2 tunnels with different endpoints (one of ISP1 and other with ISP2)
Or if is possible to have 2 WG and use the same IP for the WG tunnel.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Multiple Wireguard tunnels using different WAN links

Mon Oct 24, 2022 8:21 pm

So what you actually need is that your Mikrotik responds via the same WAN to which the initial request has arrived. So read this post, starting from the last paragraph that links it to your application scenario, and come back here if something is not clear there. Just bear in mind that here we deal with own traffic of the Mikrotik, so the translation of connection-mark to routing-mark must be done in chain output of mangle.

For your use case, there is no need to bind a separate instance of Wireguard to each WAN interface, you can use a single common one. But there is an important point, you have to activate the keepalive for the peers, for two reasons:
  • as the UDP connections have a lifetime of 3 minutes by default so if there was no traffic in either direction for longer than that, the firewall woud forget about the connection. And if the first Wireguard transport packet after the connection has been forgotten is sent by the Mikrotik, it is sent using routing table main.
  • the firewall/NAT at the "client" side will also forget the connection after some time if there is no traffic in either direction
It is an interesting approach for an ISP not to allow its customers with public IPs to talk to each other. Is this only true for customers connected in the same geographical area, who maybe get public IPs from the same subnet and thus port isolation on the access network makes some sense, or does this happen even if you connect from another city but using the same ISP?
 
iNaik
just joined
Posts: 8
Joined: Mon Oct 24, 2022 2:33 pm

Re: Multiple Wireguard tunnels using different WAN links

Tue Oct 25, 2022 10:41 am

Hmmm, okey.
I think that I understand a bit more.

I need to configure mikoritk to respond over the same ISP that arrives the request. Changing the routing mark in output chain.

But I'm not very clear how to intercept the input from ISP2 and respond with the same ISP2 the wireguard link. I will try some things.

Very thanks Niks
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Multiple Wireguard tunnels using different WAN links

Tue Oct 25, 2022 10:47 am

But I'm not very clear how to intercept the input from ISP2 and respond with the same ISP2 the wireguard link.
It's described in the post I've linked - when processing the initial packets of connections, you assign a connection-mark "via WAN 2" to all connections whose initial packet came in via WAN 2. And in the output chain, you assign a routing mark "via WAN 2" to all packets belonging to connections whose connection-mark is "via WAN 2".
 
iNaik
just joined
Posts: 8
Joined: Mon Oct 24, 2022 2:33 pm

Re: Multiple Wireguard tunnels using different WAN links

Tue Oct 25, 2022 11:57 am

Like this.

1. Take the input from WAN2 with WG port (13231) and make a connection mark
2. Take the connection mark and mark packets
3. Take the marked packets to routing mark with WAN2

It seems to be working with the frist two, but no on "out" chain.
You do not have the required permissions to view the files attached to this post.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Multiple Wireguard tunnels using different WAN links

Tue Oct 25, 2022 1:05 pm

Forget about packet-mark. Just assign connection-mark to packets matching connection-state=new in-interface=WAN2 in prerouting, and just assign the routing-mark to packets bearing that connection-mark in output.

The packet-mark and routing-mark are only valid for the packet to which they have been assigned; the connection-mark is inherited by all packets belonging to the same connection, regardless their direction.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 19107
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: Multiple Wireguard tunnels using different WAN links

Tue Oct 25, 2022 10:30 pm

WHY? SINDY you have put requirements in the OPs mouth.
Also has it been confirmed that both WANIP are public and accessible from external clients? If not then one cannot use that particular WAN for wireguard!!

In other words, the requirement is not clearly explained when you say wireguard do yo mean incoming clients only?
Do you mean two connections to a third party VPN provider?

For example if its simply for incoming wireguard traffic, I potentially disagree with Sindy, in that there may be no need for any mangling or fancy routes..........
IF wan1 is not available the customer/client at the remote site goes to the second wireguard interface at the client/remote site associated with the second wireguard interface on the router side.

For example on my IOS Phone I can create multiple Wireguard profiles.........

SO
at the main router..........
input chain dst-port=14441 in-interface=WAN1
input chain dst-port=15552 in-interface=WAN2

/IP address
add address= 10.10.44.1/24 interface=WG1
add address=10.50.55.1/24 interface=WG2

ETC.......

Simple as pie!
The Routes are auto created by MT,
The allowed IPs on the router are similar

first wireguard interface=WG1 listening port=14441
wg1 peers
allowed IPs=10.10.44.2/32

second wireguare interface=WG2 listening port=15551
wg2 peers
allowed IPs=10.10.55.2/32

Remote SITE
Client1 Setup
Interface info: Wireguard-IP address = 10.10.44.2/32
Peer info: endpoint/port = WAN1IP/dydns url : 14441 keep alive=30 seconds

Client 2 Setup
Interface info: Wireguard-IP address=10.10.55.2/32
Peer info: endpoint/port = WAN2IP/dyndns url : 15551 keep alive=35 seconds
 
iNaik
just joined
Posts: 8
Joined: Mon Oct 24, 2022 2:33 pm

Re: Multiple Wireguard tunnels using different WAN links

Fri Oct 28, 2022 8:05 pm

Hi anav,

Thanks for info,
The problem of the ISP without connection between clients , is a porblem that we are working with the ISP
Lets imagine that the client network and my network are correctly accesible.
Now I'm trying to connect with 4G connection. I try with DDNS of ISP1 and DDNS of ISP2. Different DDNS but using the same port (13231).

The question is if is possible to have 1 wireguard interface on Mikrotik and can be used with the 2 WANs that I have. Changing the DDNS of the WG or creating 2 wireguards tunnels (identical except endpoint)

In the image is a diagram how is connected and the 0.0.0.0/0 routes (main [ISP1 route ] and other route [ISP2 route] and image of Main Route (ISP1) and Second Route (ISP2) on mikrotik.

I think that I need to configure something that, when the WG starts from WAN2, the response need to get out of WAN2. (now is responding with WAN1 and didn't connect)
Tried with mangle but I can't find how to change the output of a connection that started from WAN2. (image how I have configured)
You do not have the required permissions to view the files attached to this post.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Multiple Wireguard tunnels using different WAN links

Fri Oct 28, 2022 9:03 pm

Rather than posting a ton of screenshots, post the output of /export among [code] and [/code] tags (or use the [</>] button), after removing the serial number and anonymising any public IP addresses and login names to services.

I hazily remember there were some issues in earlier ROS 7 versions with handling Wireguard transport packets by mangle chain output. What ROS version are you running?

In any case, I'd suggest to open a [Terminal] window in Winbox, make it as wide as your screen allows, run /tool sniffer quick port=13231 in it, and make a connection attempt from the client. You should see the incoming request and the outgoing response in the sniff along with the interfaces they go through.

If you can see the request to come in but no response at all, there may be a firewall issue. If you can see the response to leave but via the wrong WAN interface, something is wrong with the routing mark setting.
 
iNaik
just joined
Posts: 8
Joined: Mon Oct 24, 2022 2:33 pm

Re: Multiple Wireguard tunnels using different WAN links

Sat Oct 29, 2022 12:10 pm

Sorry for the amount of pics, normally I use the graphical more than the terminal (but also working with terminal commands)

My version is of RouterOS is 7.6
In the pic of sniffer (i don't know how to see on terminal for copy the code). There is a request from CLIENT to WAN2, and a reponse from WAN1 to client.
I've some firewall configs but there are for other ports (in NAT), and like blocks of IPs on filters (checked that there is no block IPs that I'm using), and some mangle (for changing the route for some address list).

With the "mark connection" and "mark routing" for change the route of a connection that is incomming from WAN2 to reply with WAN2, the "mark routing" with chain "output" didn't have traffic of packets, but the "mark connection" recives packets when I start the WG from a client with WAN2 configured.
/ip firewall mangle 
chain=prerouting action=mark-connection new-connection-mark=WG-VODA-conn connection-state=new protocol=udp in-interface=pppoe-VODAFONE dst-port=13231
chain=output action=mark-routing  new-routing-mark=rFIBRA-VODA  connection-mark=WG-VODA-conn
Any idea to resolve?
You do not have the required permissions to view the files attached to this post.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Multiple Wireguard tunnels using different WAN links

Sat Oct 29, 2022 3:53 pm

Sorry for the amount of pics, normally I use the graphical more than the terminal (but also working with terminal commands)
I don't ask you to change your working habits, I only ask you to provide the configuration in a form that is complete and way more efficient than screenshots in terms of bits of information per pixel.

There is a request from CLIENT to WAN2, and a reponse from WAN1 to client.
So there is indeed an issue with assigning the routing-mark. If it was a filter issue, the response would not be there at all. I cannot see whether there is an additional match condition on one of the other tabs, so it may be both a bug or a configuration issue; the export (or a screenshot of every single tab for that rule) would remove this doubt.

So remove (or just disable) the action=mark-routing rule in chain output, and instead add the following routing rule:
/routing/rule/add src-address=ip.of.wan.2 action=lookup table=rFIBRA-VODA
(assuming it will be the only routing rule in your configuration - the mutual order of rules matters so if it is not, they may need to be rearranged).

The solution with connection-mark and its translation to routing-mark in mangle is more generic and can be fine-tuned if necessary; the solution with routing rule is more rudimental but sufficient for this particular scenario. It should either resolve the issue or at least show that there is indeed a bug in handling of outgoing Wireguard transport packets.

Regarding missing MAC addresses on PPPoE interface, this is normal - PPPoE is an L3 interface so the IP packets that flow through it have no Ethernet header.
 
iNaik
just joined
Posts: 8
Joined: Mon Oct 24, 2022 2:33 pm

Re: Multiple Wireguard tunnels using different WAN links

Mon Oct 31, 2022 1:21 pm

Hi,

With the code
routing/rule/add src-address=ip.of.wan.2 action=lookup table=rFIBRA-VODA
the connection with wireguard and WAN2, works perfect! I assume that this rule will not affect to normal usage.

Any way to do with filters/NAT/mangle? If is better.

Very very thanks!!!!!
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Multiple Wireguard tunnels using different WAN links

Mon Oct 31, 2022 1:37 pm

This way all requests that arrive via WAN 2 to the router itself are responded via WAN 2.

For port forwarding to some hosts in LAN, you would need the way with mangle rules if you wanted granularity by protocols and ports or if you wanted the responses from the same LAN host to be routed via the same WAN through which the request has arrived.

The router itself responds the requests from the address to which the request has arrived, so you can use the source address as a key to choose the routing table. For requests that are forwarded to the same address in LAN regardless through which WAN they have arrived, the response always comes from the same address, so they are undistinguishavle from each other, and the connection-mark has to be used.

Who is online

Users browsing this forum: Ahrefs [Bot] and 53 guests